Geoff,
I added a ip to cloudbr0 and now I see a ARP in my management
router for the private address. I still can't ping them from the
management lan. Still the same issue as before except I notice that my
console proxy and ssvm have different routes. Neither of the system vm's
can reach the internet but can ping the public gateway but nothing
farther. Note I tried with GRE instead of vlans to see if this made a
difference. Is there firewall setting on the bridges / physicals that I
am missing? Really lost now.
cloudbr0 = private 172.16.0.0/16
cloudbr1 = guest
cloudbr2 = public 216.249.111.0/24
ssvm
root@s-2-VM:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2
8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1
8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3
216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
Console Proxy
root@v-1-VM:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2
8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1
8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
Yes I will add a ip to the cloudbr0 and see if that fixes the problem. It's a
good time since I just reset the db to start over. I don't want to vlan so can
I use advanced networking with GRE instead? GRE takes no switch configuration
right?
Sent from my Galaxy S®III
-------- Original message --------
From: Geoff Higginbottom <geoff.higginbot...@shapeblue.com>
Date:04/08/2014 4:17 PM (GMT-05:00)
To: "<users@cloudstack.apache.org>" <users@cloudstack.apache.org>
Subject: Re: System vm's with wrong network routing
Matthew,
Your Bridge configs look OK, but we normally see the management IP on the
bridge.
Can you move your clustered file system onto a dedicated nic, allowing
management IP to be placed on the bridge, this may stop the self fencing issues.
It seems too much of a coincidence that the management IP is not on the bridge
and it's the management traffic which is failing.
Regards
Geoff Higginbottom
CTO / Cloud Architect
D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 0540<tel:+442036030540> | M:
+447968161581<tel:+447968161581>
geoff.higginbot...@shapeblue.com<mailto:geoff.higginbot...@shapeblue.com> |
www.shapeblue.com<htp://www.shapeblue.com/>
ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N
4HS<x-apple-data-detectors://37>
On 8 Apr 2014, at 19:56, "Matthew Midgett"
<supp...@trickhosting.biz<mailto:supp...@trickhosting.biz>> wrote:
Here are my interfaces and I am sure the labels are correct. Any other ideas?
This is for my management ip, I couldn't put it on the bridge as my clustered
file system would keep fencing it self.
cat ifcfg-eth0
DEVICE="eth0"
BOOTPROTO=static
HWADDR="78:E7:D1:8E:2F:AE"
NM_CONTROLLED="none"
ONBOOT=yes
TYPE="Ethernet"
IPADDR=172.16.0.11
NETMASK=255.255.0.0
GATEWAY=172.16.0.1
This is where the physicals for my bridges start
cat ifcfg-eth1
DEVICE=eth1
BOOTPROTO=none
HWADDR=78:E7:D1:8E:2F:B0
ONBOOT=yes
USERCTL=no
NM_CONTROLLED=no
BRIDGE=cloudbr0
cat ifcfg-cloudbr0
DEVICE=cloudbr0
NM_CONTROLLED=no
ONBOOT=yes
TYPE=Bridge
NAME=cloudbr0
cat ifcfg-eth2
DEVICE=eth2
BOOTPROTO=none
HWADDR=78:E7:D1:8E:2F:B2
ONBOOT=yes
USERCTL=no
BRIDGE=cloudbr1
NM_CONTROLLED=no
cat ifcfg-cloudbr1
DEVICE=cloudbr1
NM_CONTROLLED=no
ONBOOT=yes
TYPE=Bridge
NAME=cloudbr1
cat ifcfg-eth3
DEVICE=eth3
BOOTPROTO=none
HWADDR=78:E7:D1:8E:2F:B4
ONBOOT=yes
USERCTL=no
BRIDGE=cloudbr2
NM_CONTROLLED=no
cat ifcfg-cloudbr2
DEVICE=cloudbr2
NM_CONTROLLED=no
ONBOOT=yes
TYPE=Bridge
BOOTPROTO=none
NAME=cloudbr2
The interfaces match my layout and the physical network
cloudbr0 - Management
cloudbr1 - guest
cloudbr2 - public
brctl show
bridge name bridge id STP enabled interfaces
cloud0 8000.fe00a9fe01e4 no vnet0
vnet3
cloudbr0 8000.78e7d18e2fb0 no eth1
vnet1
vnet4
vnet6
cloudbr1 8000.78e7d18e2fb2 no eth2
cloudbr2 8000.78e7d18e2fb4 no eth3
vnet2
vnet5
virbr0 8000.5254007e4d34 yes virbr0-nic
On 04/08/2014 02:28 PM, Geoff Higginbottom wrote:
I would check the traffic labels for the management network match for your zone
and hosts.
Regards
Geoff Higginbottom
CTO / Cloud Architect
D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 0540<tel:+442036030540> | M:
+447968161581<tel:+447968161581>
geoff.higginbot...@shapeblue.com<mailto:geoff.higginbot...@shapeblue.com><mailto:geoff.higginbot...@shapeblue.com>
| www.shapeblue.com<http://www.shapeblue.com><htp://www.shapeblue.com/> |
Twitter:@cloudstackguru<https://twitter.com/#!/cloudstackguru>
ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N
4HS<x-apple-data-detectors://5>
On 8 Apr 2014, at 17:44, "Matthew Midgett"
<supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp...@trickhosting.biz>>
wrote:
My problem is that the ssvm will not ping anything on the management network,
It will not do dns lookups as it will not ping past the default public gateway.
The ssvm is up and I can ping the public ip from a different subnet.
If i remove this I can use external dns and the ssvm can ping anything public
8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1
8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1
I change the values as Geoff had suggested and I rebooted the management
service. Then i destroyed the ssvm so it would be recreated with the new
changes. Here is the route as it is now and not working.
root@s-9-VM:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2
8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1
8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3
216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
To me it doesn't look like the use external dns setting to true didn't do
anything. It sill wants to tell the dns to go across the management network.
That would be fine if the routes to the management network worked.
I'm willing to pay someone to help me at this point.
On 04/08/2014 12:10 PM, Erik Weber wrote:
You don't need gateway for network in the same subnet on an interface.
You never told us what your real problem is (i think), so why not start by
telling us what is not working :-)
As i tried to explain, 8.8.8.8 and 8.8.4.4 will be routed over the mgmt
interface if it's entered as the internal dns. If that is your problem, try
changing the value Geoff suggested.
Erik
8. apr. 2014 17:10 skrev "Matthew Midgett"
<supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp...@trickhosting.biz>>
følgende:
Erik that would be fine but this is wrong, this means its going to route
over the public address as 0.0.0.0 points to my public gateway.
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3
On 04/08/2014 11:03 AM, Erik Weber wrote:
If you use the same dns servers for both internal and external it will add
a route over mgmt interface.
Erik Weber
8. apr. 2014 16:07 skrev "Matthew Midgett"
<supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp...@trickhosting.biz>>
følgende:
I destroyed the VM so it would create a new route. First off the 8.8.8.8
and 8.8.4.4 should have a default gateway of 0.0.0.0 if its to use the
public address to get dns. If its supposed to use the management network
then the 172.16.0.0 0.0.0.0 should be 172.16.0.0 172.16.0.1
This way it doesn't route properly
root@s-4-VM:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2
8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1
8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3
216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
This way works.
root@s-4-VM:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
172.16.0.0 172.16.0.1 255.255.0.0 U 0 0 0
eth1
172.16.0.0 172.16.0.1 255.255.0.0 U 0 0 0
eth3
216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
Still not sure where I should put this. I know its the way that Im
connecting to it but what route should it take? I'm thinking it should be
172.16.0.0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
On 04/08/2014 01:35 AM, Geoff Higginbottom wrote:
Matthew,
Can you give examples of the routes you are seeing and explain why they
are wrong please.
Regards
Geoff Higginbottom
CTO / Cloud Architect
D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 0540<tel:
+442036030540> | M: +447968161581<tel:+447968161581>
geoff.higginbot...@shapeblue.com<mailto:geoff.higginbot...@shapeblue.com><mailto:geoff.higginbot...@shapeblue.com><mailto:geoff.higginbottom@
shapeblue.com<http://shapeblue.com><http://shapeblue.com>>
|
www.shapeblue.com<http://www.shapeblue.com><http://www.shapeblue.com><htp://www.shapeblue.com/>
| Twitter:@cloudstackguru
<
https://twitter.com/#!/cloudstackguru>
ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N
4HS<x-apple-data-detectors://5>
On 7 Apr 2014, at 22:24, "Matthew Midgett"
<supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp...@trickhosting.biz><
mailto:supp...@trickhosting.biz>> wrote:
My system vm's are being created with the wrong routes. Manually
deleting
and adding them fixes the problem. Where would I go to fix this
permanently?
Sent from my Galaxy S(r)III
Need Enterprise Grade Support for Apache CloudStack?
Our CloudStack Infrastructure Support<http://shapeblue.com/
cloudstack-infrastructure-support/> offers the best 24/7 SLA for
CloudStack Environments.
Apache CloudStack Bootcamp training courses
**NEW!** CloudStack 4.2.1 training<http://shapeblue.com/
cloudstack-training/>
28th-29th May 2014, Bangalore. Classromm<http://shapeblue.
com/cloudstack-training/>
16th-20th June 2014, Region A. Instructor led, On-line<
http://shapeblue.com/cloudstack-training/>
23rd-27th June 2014, Region B. Instructor led, On-line<
http://shapeblue.com/cloudstack-training/>
15th-20th September 2014, Region A. Instructor led, On-line<
http://shapeblue.com/cloudstack-training/>
22nd-27th September 2014, Region B. Instructor led, On-line<
http://shapeblue.com/cloudstack-training/>
1st-6th December 2014, Region A. Instructor led, On-line<
http://shapeblue.com/cloudstack-training/>
8th-12th December 2014, Region B. Instructor led, On-line<
http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are
intended
solely for the use of the individual to whom it is addressed. Any views
or
opinions expressed are solely those of the author and do not necessarily
represent those of Shape Blue Ltd or related companies. If you are not
the
intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the
sender
if you believe you have received this email in error. Shape Blue Ltd is
a
company incorporated in England & Wales. ShapeBlue Services India LLP
is a
company incorporated in India and is operated under license from Shape
Blue
Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
Brasil
and is operated under license from Shape Blue Ltd. ShapeBlue is a
registered trademark.
This email and any attachments to it may be confidential and are intended solely
for the use of the individual to whom it is addressed. Any views or opinions
expressed are solely those of the author and do not necessarily represent those of
Shape Blue Ltd or related companies. If you are not the intended recipient of this
email, you must neither take any action based upon its contents, nor copy or show
it to anyone. Please contact the sender if you believe you have received this email
in error. Shape Blue Ltd is a company incorporated in England & Wales.
ShapeBlue Services India LLP is a company incorporated in India and is operated
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue
is a registered trademark.
This email and any attachments to it may be confidential and are intended solely
for the use of the individual to whom it is addressed. Any views or opinions
expressed are solely those of the author and do not necessarily represent those of
Shape Blue Ltd or related companies. If you are not the intended recipient of this
email, you must neither take any action based upon its contents, nor copy or show
it to anyone. Please contact the sender if you believe you have received this email
in error. Shape Blue Ltd is a company incorporated in England & Wales.
ShapeBlue Services India LLP is a company incorporated in India and is operated
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue
is a registered trademark.
