OK, this is what I got - 2x External network (flat or fixed vlan?) - Tenant network: That allows for connecting to other external servers on the same l2 net + gre --> I assume you have a fixed vlan that you need to use, right? --> In addition you want to have gre tenant networks?
OK, now coming to the requirements: Each l2 network requires its own interface. Background: for l2 networking (flat/vlan) the physical port is plugged into the vswitch. Obviously a port can only be plugged in one vswitch at once. So you need at least 2 interfaces for your external networks and one for your tenant vlan network. By interface I mean either an ethx interface or a vlan device. But let's start with the tenant network! Don't care about the external networks at first. Upfront: I'm not an expert - I only have experience with a single type of tenant networks. So this is what I can explain. There might be other options that I'm not aware of You have 3 choices, it depends on what you need: 1) you want multiple gre & 1 vlan network for tenants - Use gre for tenant networks - Create a single provider network that matches your l2 network 2) you want multiple vlan networks for tenants - use vlan for tenant networks (vlan ids will be picked dynamically along a range you provide) 3) You just need one tenant network, that matches your specific vlan id - Create a single provider network that matches your l2 network - Or use vlan tenant networks providing a range with a single vlan id Which one would be the right way? On Di, 2015-06-30 at 13:51 +0000, Yngvi Páll Þorfinnsson wrote: > Hi Andreas, > > Our plan is (or was) to do the following. > > Connect the servers with two physical interfaces, configured into a bound > interface in order to provide failover resilance. Use this bound interface to > provide all connectivity for the node (management, ext nets and Tennants). > Additionally we wish to configure multible (at least two) external networks. > One to connect to the open internet and the other to connect to trusted > internal network within the company. > And at last we wish the have the Tennant network configured as VLAN (at least > as an option in addition to GRE and VXLAN if possible). The VLAN config is > needed, since we also need (in addition to the extra external networks) to be > able to create a Tennant subnet that connects on L2 to physical servers. > > After working on this, with great help from you, it seems to be rather > difficult to get this running:-) So perhaps we need to rethink our plans? It > also seems to be an security issue to use the same physical interface to > provide ext, tennant and management networks. > > As to implement VLANs I configured > > root@network2:/# cat /proc/net/vlan/config > VLAN Dev name | VLAN ID > Name-Type: VLAN_NAME_TYPE_RAW_PLUS_VID_NO_PAD > bond0.48 | 48 | bond0 > bond0.47 | 47 | bond0 > bond0.45 | 45 | bond0 > > bond0.48 -> mgtm network > bond0.47 -> tunnel network > bond0.45 -> exteral network no.1 > > currently only one external network has been configured. > > Best regards > Yngvi > > -----Original Message----- > From: Andreas Scheuring [mailto:scheu...@linux.vnet.ibm.com] > Sent: 30. júní 2015 12:36 > To: Yngvi Páll Þorfinnsson > Cc: openstack@lists.openstack.org > Subject: Re: [Openstack] error creating instance > > Ok, that was not clear to me! I thought your intention was, to have the data > network as vlan (which you configured with vlan 47 or something like that on > top of your bond) and then use gre on this vlan for tenant networks. > > So you want to have one network that uses vlan 47? Or do you want to have the > cloudy thing, that you can create as many network as you like each with an > increasing vlan id? > > > So let's exactly define what you're looking for, before making other > suggestions ;) > > > I came to this gre assumption, as all your openvswitch-agent config files > still have gre configured! (see ml2.conf - always tunneling > enabled) > > > To see where we are, could you please authenticate as admin and dump > > > neutron net-show demo-net1 > > > > > > On Di, 2015-06-30 at 11:22 +0000, Yngvi Páll Þorfinnsson wrote: > > HI Andreas > > > > Just to make it clear, > > we need to have our tenant networks as VLANs, in order to have native > > connection from VM to physical host on our server networks. > > > > Best regards > > Yngvi > > > > -----Original Message----- > > From: Yngvi Páll Þorfinnsson > > Sent: 30. júní 2015 08:15 > > To: Andreas Scheuring > > Cc: openstack@lists.openstack.org > > Subject: Re: [Openstack] error creating instance > > > > Well, I don't have the dashboard yet. I've installed, but I'm waiting for > > access from the network guys... > > > > I'm now in a kind of dilemme, trying to delete the network; > > > > root@controller2:/# neutron port-list > > +--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+ > > | id | name | mac_address | > > fixed_ips > > | > > +--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+ > > | 0bcdb649-28cc-47d3-9a84-62b9aece589e | | fa:16:3e:9c:cb:6d | > > {"subnet_id": "c40fa8e3-cd8e-4566-ade6-5f3eabed121c", "ip_address": > > "157.157.8.51"} | > > | 231895ce-4d1f-4fba-a5e7-8dc12843cb3e | | fa:16:3e:97:68:fe | > > {"subnet_id": "2c79bb00-0ace-4319-8151-81210ee3dfb2", "ip_address": > > "172.22.18.3"} | > > | f371a118-8270-496a-9868-5e921949094d | | fa:16:3e:eb:1c:29 | > > {"subnet_id": "2c79bb00-0ace-4319-8151-81210ee3dfb2", "ip_address": > > "172.22.18.1"} | > > +--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+ > > root@controller2:/# > > root@controller2:/# neutron router-list > > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+ > > | id | name | > > external_gateway_info > > > > | distributed | ha | > > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+ > > | 80dd1cff-a530-43c2-97e8-7cb331e928cb | demo-router | {"network_id": > > "b43da44a-42d5-4b1f-91c2-d06a923deb29", "enable_snat": true, > > "external_fixed_ips": [{"subnet_id": > > "c40fa8e3-cd8e-4566-ade6-5f3eabed121c", "ip_address": "157.157.8.51"}]} | > > False | False | > > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+ > > root@controller2:/# > > root@controller2:/# neutron port-delete > > 0bcdb649-28cc-47d3-9a84-62b9aece589e > > Port 0bcdb649-28cc-47d3-9a84-62b9aece589e has owner > > network:router_gateway and therefore cannot be deleted directly via > > the port API. (HTTP 409) (Request-ID: > > req-0aff43f0-5650-4a9c-82c0-211fb6fcebb6) > > root@controller2:/# neutron router-delete > > 80dd1cff-a530-43c2-97e8-7cb331e928cb > > Router 80dd1cff-a530-43c2-97e8-7cb331e928cb still has ports (HTTP 409) > > (Request-ID: req-bf34e06f-41e8-419a-8e5a-00061cd9d5f9) > > root@controller2:/# > > > > > > 2) The demo network > > > > Oh, I just followed the manual, so it's like this: > > > > source demo-openrc.sh > > neutron net-create demo-net > > neutron subnet-create demo-net --name demo-subnet \ --gateway x.y.z.1 > > x.y.z.0/24 > > > > And the router was also created according to the manual; > > > > neutron router-create demo-router > > neutron router-interface-add demo-router demo-subnet > > > > and then is was attached to the external net: > > > > Attach the router to the external network by setting it as the gateway: > > > > neutron router-gateway-set demo-router ext_net1101 > > > > Best regards > > Yngvi > > > > -----Original Message----- > > From: Andreas Scheuring [mailto:scheu...@linux.vnet.ibm.com] > > Sent: 30. júní 2015 08:01 > > To: Yngvi Páll Þorfinnsson > > Cc: openstack@lists.openstack.org > > Subject: Re: [Openstack] error creating instance > > > > Ok, > > > > The best thing is to login via the dashboard. Go to your router and remove > > the interface to the internal network. There's also an api call, but I > > don't know it from the top of my head. > > Then try to remove the network again. > > > > Before recreating it: > > > > How did you create your demo-network? Did you specify to have a vlan > > network? If NOT, you should update the following configuration on your > > controller node tenant_network_types = vlan,gre --> > > tenant_network_types = gre This ensures, that only gre network are > > being created (I'm not 100% sure how this behehaves with multiple > > entries here...) > > > > Restart your neutron-server > > > > Create a network without specifiying anything about gre or vlan or > > something else. > > > > Your gre network then will created and used within your static host vlan > > you configured (vlan 47). > > > > > > Andreas > > > > > > > > On Di, 2015-06-30 at 07:53 +0000, Yngvi Páll Þorfinnsson wrote: > > > OK Andreas, I have different id's now on the networks, Since I > > > dropped neutron db yesterday, and created again, ( as well as the > > > networks) > > > > > > But it's the same setup, and now the demo network is like this: > > > > > > root@controller2:/# neutron net-list > > > +--------------------------------------+-------------+-----------------------------------------------------+ > > > | id | name | subnets > > > | > > > +--------------------------------------+-------------+-----------------------------------------------------+ > > > | b43da44a-42d5-4b1f-91c2-d06a923deb29 | ext_net1101 | > > > c40fa8e3-cd8e-4566-ade6-5f3eabed121c 157.157.8.0/24 | > > > | 3446e54b-346f-45e5-89a2-1ec4eef251ab | demo-net | > > > 2c79bb00-0ace-4319-8151-81210ee3dfb2 172.22.18.0/24 | > > > +--------------------------------------+-------------+-----------------------------------------------------+ > > > root@controller2:/# > > > root@controller2:/# neutron net-show > > > 3446e54b-346f-45e5-89a2-1ec4eef251ab > > > +---------------------------+--------------------------------------+ > > > | Field | Value | > > > +---------------------------+--------------------------------------+ > > > | admin_state_up | True | > > > | id | 3446e54b-346f-45e5-89a2-1ec4eef251ab | > > > | name | demo-net | > > > | provider:network_type | vlan | > > > | provider:physical_network | external | > > > | provider:segmentation_id | 1102 | > > > | router:external | False | > > > | shared | False | > > > | status | ACTIVE | > > > | subnets | 2c79bb00-0ace-4319-8151-81210ee3dfb2 | > > > | tenant_id | f976b7c713c64b028c0e89f3956795ed | > > > +---------------------------+--------------------------------------+ > > > root@controller2:/# > > > > > > and this is the one I should delete then ( and re-create). > > > > > > I'm having difficulties deleting the network ; > > > > > > root@controller2:/# source demo-openrc.sh root@controller2:/# > > > root@controller2:/# neutron net-delete > > > 3446e54b-346f-45e5-89a2-1ec4eef251ab > > > Unable to complete operation on network > > > 3446e54b-346f-45e5-89a2-1ec4eef251ab. There are one or more ports > > > still in use on the network. (HTTP 409) (Request-ID: > > > req-432fee24-e554-42fb-bfc4-f9d597903eea) > > > root@controller2:/# > > > root@controller2:/# neutron router-list > > > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > > > | id | name | > > > external_gateway_info > > > > > > | > > > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > > > | 80dd1cff-a530-43c2-97e8-7cb331e928cb | demo-router | {"network_id": > > > | "b43da44a-42d5-4b1f-91c2-d06a923deb29", "enable_snat": true, > > > | "external_fixed_ips": [{"subnet_id": > > > | "c40fa8e3-cd8e-4566-ade6-5f3eabed121c", "ip_address": > > > | "157.157.8.51"}]} | > > > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > > > root@controller2:/# > > > root@controller2:/# neutron router-delete > > > 80dd1cff-a530-43c2-97e8-7cb331e928cb > > > Router 80dd1cff-a530-43c2-97e8-7cb331e928cb still has ports (HTTP > > > 409) > > > (Request-ID: req-a90df408-2551-4f18-86d4-263daa325307) > > > > > > But , nova list is empty; > > > > > > root@controller2:/# nova list > > > +----+------+--------+------------+-------------+----------+ > > > | ID | Name | Status | Task State | Power State | Networks | > > > +----+------+--------+------------+-------------+----------+ > > > +----+------+--------+------------+-------------+----------+ > > > root@controller2:/# > > > > > > > > > Best regards > > > Yngvi > > > > > > -----Original Message----- > > > From: Andreas Scheuring [mailto:scheu...@linux.vnet.ibm.com] > > > Sent: 30. júní 2015 06:43 > > > To: Yngvi Páll Þorfinnsson > > > Cc: openstack@lists.openstack.org > > > Subject: Re: [Openstack] error creating instance > > > > > > Now all makes absolutely sense: > > > > > > > > > +---------------------------+--------------------------------------+ > > > | Field | Value | > > > +---------------------------+--------------------------------------+ > > > | admin_state_up | True | > > > | id | 7a344656-815c-4116-b697-b52f9fdc6e4c | > > > | name | demo-net | > > > | provider:network_type | vlan | > > > | provider:physical_network | external | > > > | provider:segmentation_id | 1102 | > > > | router:external | False | > > > | shared | False | > > > | status | ACTIVE | > > > | subnets | c57880bf-2869-4b1d-b31c-0d2d083d05e4 | > > > | tenant_id | f976b7c713c64b028c0e89f3956795ed | > > > +---------------------------+--------------------------------------+ > > > > > > This is you demo network. It's mapped to the pyhsical network "external" > > > (defined in bridgemapping) and uses vlan. > > > > > > But that the wrong way. If I get you right, you still want to use gre but > > > now over your static vlan, right? So what you need to do is, to delete > > > your demo-net. > > > > > > > neutron net-delete 7a344656-815c-4116-b697-b52f9fdc6e4c > > > > > > And just create a new one without specifying any provider attributes > > > > > > > neutron net-create ... > > > Not sure, but I guess it's sufficient to specify a name! > > > > > > > neutron net-show <your-new-net> > > > > > > It should look somehting like this: > > > > > > --> no pyhsical network > > > --> network_type = gre > > > +---------------------------+--------------------------------------+ > > > > | Field | Value | > > > > +---------------------------+--------------------------------------+ > > > > | admin_state_up | True | > > > > | id | ef6552a5-be39-4bcc-9dde-2a200eaca64d | > > > > | mtu | 0 | > > > > | name | private | > > > > | provider:network_type | vxlan | > > > > | provider:physical_network | | > > > > | provider:segmentation_id | 1001 | > > > > | router:external | False | > > > > | shared | False | > > > > | status | ACTIVE | > > > > | subnets | 4b539feb-b104-4f69-83ba-76f746a2c592 | > > > > | | ac255618-afe9-4aea-b86d-b662b68e9d9d | > > > > | tenant_id | 3c4ddcff52a74f2b97b71392300aa74d | > > > > +---------------------------+--------------------------------------+ > > > > > > > > > > > > If you want to switch over totally to vlan (not using gre in your static > > > vlan anymore), you have to reconfigure the ml2 config files. But I'm not > > > sure, if dynamic Openstack vlans nested into a static host vlan work out. > > > But let's discuss this if relevant. > > > > > > > > > You can also ping me on irc (freenode, #openstack, to speed up debugging. > > > My nic name is 'scheuran'. > > > > > > > > > > > > Andreas > > > > > > > > > On Mo, 2015-06-29 at 15:54 +0000, Yngvi Páll Þorfinnsson wrote: > > > > OK, > > > > > > > > This is the network list > > > > > > > > root@controller2:/# neutron net-list > > > > +--------------------------------------+-------------+-----------------------------------------------------+ > > > > | id | name | subnets > > > > | > > > > +--------------------------------------+-------------+-----------------------------------------------------+ > > > > | 1cb21927-6996-4022-8d3e-292390dad6d0 | ext_net1101 | > > > > 7b84ec73-eaa7-479f-872f-41c6cb2083c5 157.157.8.0/24 | > > > > | 7a344656-815c-4116-b697-b52f9fdc6e4c | demo-net | > > > > c57880bf-2869-4b1d-b31c-0d2d083d05e4 172.22.18.0/24 | > > > > +--------------------------------------+-------------+-----------------------------------------------------+ > > > > root@controller2:/# > > > > root@controller2:/# > > > > root@controller2:/# neutron net-show > > > > 1cb21927-6996-4022-8d3e-292390dad6d0 > > > > +---------------------------+--------------------------------------+ > > > > | Field | Value | > > > > +---------------------------+--------------------------------------+ > > > > | admin_state_up | True | > > > > | id | 1cb21927-6996-4022-8d3e-292390dad6d0 | > > > > | name | ext_net1101 | > > > > | provider:network_type | vlan | > > > > | provider:physical_network | external | > > > > | provider:segmentation_id | 1101 | > > > > | router:external | True | > > > > | shared | False | > > > > | status | ACTIVE | > > > > | subnets | 7b84ec73-eaa7-479f-872f-41c6cb2083c5 | > > > > | tenant_id | c96aec0da5e542ad8e3198aaccc498c7 | > > > > +---------------------------+--------------------------------------+ > > > > root@controller2:/# neutron net-show > > > > 7a344656-815c-4116-b697-b52f9fdc6e4c > > > > +---------------------------+--------------------------------------+ > > > > | Field | Value | > > > > +---------------------------+--------------------------------------+ > > > > | admin_state_up | True | > > > > | id | 7a344656-815c-4116-b697-b52f9fdc6e4c | > > > > | name | demo-net | > > > > | provider:network_type | vlan | > > > > | provider:physical_network | external | > > > > | provider:segmentation_id | 1102 | > > > > | router:external | False | > > > > | shared | False | > > > > | status | ACTIVE | > > > > | subnets | c57880bf-2869-4b1d-b31c-0d2d083d05e4 | > > > > | tenant_id | f976b7c713c64b028c0e89f3956795ed | > > > > +---------------------------+--------------------------------------+ > > > > root@controller2:/# > > > > > > > > and this is how I created the external network, with CMD ; > > > > > > > > neutron net-create ext_net1101 --provider:network_type vlan > > > > --provider:physical_network external --provider:segmentation_id > > > > 1101 --router:external=True > > > > > > > > > > > > But there seems not to be info on this one root@controller2:/# > > > > neutron net-show cf6489c4-7ed6-43dc-85aa-f4b8c6b501ca > > > > Unable to find network with name 'cf6489c4-7ed6-43dc-85aa-f4b8c6b501ca' > > > > > > > > > > > > best regards > > > > Yngvi > > > > > > > > > > > > -----Original Message----- > > > > From: Andreas Scheuring [mailto:scheu...@linux.vnet.ibm.com] > > > > Sent: 29. júní 2015 15:25 > > > > To: Yngvi Páll Þorfinnsson > > > > Cc: uwe.sauter...@gmail.com; openstack@lists.openstack.org > > > > Subject: Re: [Openstack] error creating instance > > > > > > > > > > > > > > > > Attempting to bind port 2bf4a49b-2ad6-4ead-a656-65814ad0724e on > > > > network 7a344656-815c-4116-b697-b52f9fdc6e4c > > > > bind_port > > > > /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_ > > > > ag > > > > en > > > > t.py:57 > > > > 2015-06-29 14:28:55.924 5328 DEBUG > > > > neutron.plugins.ml2.drivers.mech_agent > > > > [req-9fe66e60-1a70-4ad6-b21e-ef91aca8a931 None] Checking agent: > > > > {'binary': u'neutron-openvswitch-agent', 'description': None, > > > > 'admin_state_up': True, 'heartbeat_timestamp': datetime.datetime(2015, > > > > 6, 29, 14, 28, 45), 'alive': True, 'id': > > > > u'1c06fb08-105c-4659-ae0e-4a905931311e', 'topic': u'N/A', 'host': > > > > u'compute5', 'agent_type': u'Open vSwitch agent', 'started_at': > > > > datetime.datetime(2015, 6, 29, 14, 27, 45), 'created_at': > > > > datetime.datetime(2015, 6, 26, 14, 51, 14), 'configurations': > > > > {u'arp_responder_enabled': False, u'tunneling_ip': > > > > u'172.22.15.17', > > > > u'devices': 0, u'l2_population': False, u'tunnel_types': [u'gre'], > > > > u'enable_distributed_routing': False, u'bridge_mappings': {}}} > > > > bind_port > > > > /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_ > > > > ag > > > > en > > > > t.py:65 > > > > 2015-06-29 14:28:55.925 5328 DEBUG > > > > neutron.plugins.ml2.drivers.mech_openvswitch > > > > [req-9fe66e60-1a70-4ad6-b21e-ef91aca8a931 None] Checking segment: > > > > {'segmentation_id': 1102L, 'physical_network': u'external', 'id': > > > > u'cf6489c4-7ed6-43dc-85aa-f4b8c6b501ca', 'network_type': u'vlan'} > > > > for > > > > mappings: {} with tunnel_types: [u'gre'] check_segment_for_agent > > > > /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_ > > > > op > > > > en > > > > vswitch.py:52 > > > > > > > > > > > > > > > > === > > > > Checking segment: {'segmentation_id': 1102L, > > > > 'physical_network':u'external', 'id': > > > > u'cf6489c4-7ed6-43dc-85aa-f4b8c6b501ca', 'network_type': u'vlan'} > > > > for > > > > mappings: {} with tunnel_types: [u'gre'] > > > > > > > > This looks strange: Seems like your tenant network has a > > > > physical_network of type vlan assigned. That shouldn't be the case. > > > > > > > > Could you please provide the following information: > > > > > > > > Information of all Openstack networks available: > > > > > > > > > neutron net-list > > > > > > > > > neutron net-show <uuid> > > > > > > > > Especially of this one: > > > > > neturon net-show cf6489c4-7ed6-43dc-85aa-f4b8c6b501ca > > > > > > > > > > > > Usually your network should look like this (in this case vxlan): > > > > > > > > +---------------------------+--------------------------------------+ > > > > | Field | Value | > > > > +---------------------------+--------------------------------------+ > > > > | admin_state_up | True | > > > > | id | ef6552a5-be39-4bcc-9dde-2a200eaca64d | > > > > | mtu | 0 | > > > > | name | private | > > > > | provider:network_type | vxlan | > > > > | provider:physical_network | | > > > > | provider:segmentation_id | 1001 | > > > > | router:external | False | > > > > | shared | False | > > > > | status | ACTIVE | > > > > | subnets | 4b539feb-b104-4f69-83ba-76f746a2c592 | > > > > | | ac255618-afe9-4aea-b86d-b662b68e9d9d | > > > > | tenant_id | 3c4ddcff52a74f2b97b71392300aa74d | > > > > +---------------------------+--------------------------------------+ > > > > > > > > How did you create yours? via the UI? Or are you attaching your > > > > instance to the external network instead? In any case you need to > > > > attach it to your tenant network!! If it's not visible via the UI, > > > > maybe you have to switch to another tenant to get it. > > > > > > > > Hope we're close to finding the issue ;) > > > > > > > > > > > > Andreas > > > > > > > > > > > > > > > > > > -- > > Andreas > > (IRC: scheuran) > > > > > > _______________________________________________ > > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > > Post to : openstack@lists.openstack.org > > Unsubscribe : > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > > -- > Andreas > (IRC: scheuran) > > -- Andreas (IRC: scheuran) _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
