Yes I plugged my laptop in to the switch put a public on it and all is well. I have put the use . external.dns to true so I'm not sure why the dns is even in the route table.
Sent from my Galaxy S®III -------- Original message -------- From: Geoff Higginbottom <geoff.higginbot...@shapeblue.com> Date:04/10/2014 1:54 PM (GMT-05:00) To: users@cloudstack.apache.org Subject: RE: System vm's with wrong network routing Matthew You say your system VMs can ping the public gateway but nothing further, have you checked the configuration on your gateway to ensure it is setup correctly to route the return traffic back to the VMs public IPs. Regards Geoff Higginbottom D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 geoff.higginbot...@shapeblue.com -----Original Message----- From: Matthew Midgett [mailto:supp...@trickhosting.biz] Sent: 10 April 2014 18:48 To: users@cloudstack.apache.org Subject: Re: System vm's with wrong network routing Geoff, I added a ip to cloudbr0 and now I see a ARP in my management router for the private address. I still can't ping them from the management lan. Still the same issue as before except I notice that my console proxy and ssvm have different routes. Neither of the system vm's can reach the internet but can ping the public gateway but nothing farther. Note I tried with GRE instead of vlans to see if this made a difference. Is there firewall setting on the bridges / physicals that I am missing? Really lost now. cloudbr0 = private 172.16.0.0/16 cloudbr1 = guest cloudbr2 = public 216.249.111.0/24 ssvm root@s-2-VM:~# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2 8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3 216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 Console Proxy root@v-1-VM:~# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2 8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1 216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 > Yes I will add a ip to the cloudbr0 and see if that fixes the problem. It's a > good time since I just reset the db to start over. I don't want to vlan so > can I use advanced networking with GRE instead? GRE takes no switch > configuration right? > > > Sent from my Galaxy S®III > > -------- Original message -------- > From: Geoff Higginbottom <geoff.higginbot...@shapeblue.com> > Date:04/08/2014 4:17 PM (GMT-05:00) > To: "<users@cloudstack.apache.org>" <users@cloudstack.apache.org> > Subject: Re: System vm's with wrong network routing > > Matthew, > > Your Bridge configs look OK, but we normally see the management IP on the > bridge. > > Can you move your clustered file system onto a dedicated nic, allowing > management IP to be placed on the bridge, this may stop the self fencing > issues. > > It seems too much of a coincidence that the management IP is not on the > bridge and it's the management traffic which is failing. > > Regards > > Geoff Higginbottom > CTO / Cloud Architect > > > D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 > 0540<tel:+442036030540> | M: +447968161581<tel:+447968161581> > > geoff.higginbot...@shapeblue.com<mailto:geoff.higginbottom@shapeblue.c > om> | www.shapeblue.com<htp://www.shapeblue.com/> > > ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N > 4HS<x-apple-data-detectors://37> > > > > On 8 Apr 2014, at 19:56, "Matthew Midgett" > <supp...@trickhosting.biz<mailto:supp...@trickhosting.biz>> wrote: > > Here are my interfaces and I am sure the labels are correct. Any other ideas? > > This is for my management ip, I couldn't put it on the bridge as my clustered > file system would keep fencing it self. > > cat ifcfg-eth0 > DEVICE="eth0" > BOOTPROTO=static > HWADDR="78:E7:D1:8E:2F:AE" > NM_CONTROLLED="none" > ONBOOT=yes > TYPE="Ethernet" > IPADDR=172.16.0.11 > NETMASK=255.255.0.0 > GATEWAY=172.16.0.1 > > This is where the physicals for my bridges start > > cat ifcfg-eth1 > DEVICE=eth1 > BOOTPROTO=none > HWADDR=78:E7:D1:8E:2F:B0 > ONBOOT=yes > USERCTL=no > NM_CONTROLLED=no > BRIDGE=cloudbr0 > > cat ifcfg-cloudbr0 > DEVICE=cloudbr0 > NM_CONTROLLED=no > ONBOOT=yes > TYPE=Bridge > NAME=cloudbr0 > > cat ifcfg-eth2 > DEVICE=eth2 > BOOTPROTO=none > HWADDR=78:E7:D1:8E:2F:B2 > ONBOOT=yes > USERCTL=no > BRIDGE=cloudbr1 > NM_CONTROLLED=no > > cat ifcfg-cloudbr1 > DEVICE=cloudbr1 > NM_CONTROLLED=no > ONBOOT=yes > TYPE=Bridge > NAME=cloudbr1 > > cat ifcfg-eth3 > DEVICE=eth3 > BOOTPROTO=none > HWADDR=78:E7:D1:8E:2F:B4 > ONBOOT=yes > USERCTL=no > BRIDGE=cloudbr2 > NM_CONTROLLED=no > > cat ifcfg-cloudbr2 > DEVICE=cloudbr2 > NM_CONTROLLED=no > ONBOOT=yes > TYPE=Bridge > BOOTPROTO=none > NAME=cloudbr2 > > The interfaces match my layout and the physical network > > cloudbr0 - Management > cloudbr1 - guest > cloudbr2 - public > > brctl show > bridge name bridge id STP enabled interfaces > cloud0 8000.fe00a9fe01e4 no vnet0 > vnet3 > cloudbr0 8000.78e7d18e2fb0 no eth1 > vnet1 > vnet4 > vnet6 > cloudbr1 8000.78e7d18e2fb2 no eth2 > cloudbr2 8000.78e7d18e2fb4 no eth3 > vnet2 > vnet5 > virbr0 8000.5254007e4d34 yes virbr0-nic > > > > > > > > > > > On 04/08/2014 02:28 PM, Geoff Higginbottom wrote: > I would check the traffic labels for the management network match for your > zone and hosts. > > Regards > > Geoff Higginbottom > CTO / Cloud Architect > > D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 > 0540<tel:+442036030540> | M: +447968161581<tel:+447968161581> > > geoff.higginbot...@shapeblue.com<mailto:geoff.higginbottom@shapeblue.c > om><mailto:geoff.higginbot...@shapeblue.com> | > www.shapeblue.com<http://www.shapeblue.com><htp://www.shapeblue.com/> > | Twitter:@cloudstackguru<https://twitter.com/#!/cloudstackguru> > > ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N > 4HS<x-apple-data-detectors://5> > > > On 8 Apr 2014, at 17:44, "Matthew Midgett" > <supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp...@trickhosting.biz>> > wrote: > > My problem is that the ssvm will not ping anything on the management network, > It will not do dns lookups as it will not ping past the default public > gateway. The ssvm is up and I can ping the public ip from a different subnet. > > If i remove this I can use external dns and the ssvm can ping anything public > 8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 > 8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 > > I change the values as Geoff had suggested and I rebooted the management > service. Then i destroyed the ssvm so it would be recreated with the new > changes. Here is the route as it is now and not working. > > root@s-9-VM:~# route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric Ref Use Iface > 0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2 > 8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 > 8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 > 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1 > 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3 > 216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 > > To me it doesn't look like the use external dns setting to true didn't do > anything. It sill wants to tell the dns to go across the management network. > That would be fine if the routes to the management network worked. > > I'm willing to pay someone to help me at this point. > > > On 04/08/2014 12:10 PM, Erik Weber wrote: > You don't need gateway for network in the same subnet on an interface. > > You never told us what your real problem is (i think), so why not > start by telling us what is not working :-) > > As i tried to explain, 8.8.8.8 and 8.8.4.4 will be routed over the > mgmt interface if it's entered as the internal dns. If that is your > problem, try changing the value Geoff suggested. > > Erik > 8. apr. 2014 17:10 skrev "Matthew Midgett" > <supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp > o...@trickhosting.biz>> > følgende: > > Erik that would be fine but this is wrong, this means its going to > route over the public address as 0.0.0.0 points to my public gateway. > > 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1 > 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3 > > > On 04/08/2014 11:03 AM, Erik Weber wrote: > > If you use the same dns servers for both internal and external it will > add a route over mgmt interface. > > Erik Weber > 8. apr. 2014 16:07 skrev "Matthew Midgett" > <supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp > o...@trickhosting.biz>> > følgende: > > I destroyed the VM so it would create a new route. First off the > 8.8.8.8 and 8.8.4.4 should have a default gateway of 0.0.0.0 if its to > use the public address to get dns. If its supposed to use the management > network > then the 172.16.0.0 0.0.0.0 should be 172.16.0.0 172.16.0.1 > > This way it doesn't route properly > > root@s-4-VM:~# route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric Ref Use > Iface > 0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2 > 8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 > 8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 > 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1 > 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3 > 216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 > > This way works. > > root@s-4-VM:~# route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric Ref Use > Iface > 0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2 > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 > 172.16.0.0 172.16.0.1 255.255.0.0 U 0 0 0 > eth1 > 172.16.0.0 172.16.0.1 255.255.0.0 U 0 0 0 > eth3 > 216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 > > Still not sure where I should put this. I know its the way that Im > connecting to it but what route should it take? I'm thinking it should > be > 172.16.0.0 > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 > > > > > On 04/08/2014 01:35 AM, Geoff Higginbottom wrote: > > Matthew, > Can you give examples of the routes you are seeing and explain why > they are wrong please. > > Regards > > Geoff Higginbottom > CTO / Cloud Architect > > D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 0540<tel: > +442036030540> | M: +447968161581<tel:+447968161581> > > geoff.higginbot...@shapeblue.com<mailto:geoff.higginbottom@shapeblue.c > om><mailto:geoff.higginbot...@shapeblue.com><mailto:geoff.higginbottom > @ shapeblue.com<http://shapeblue.com><http://shapeblue.com>> > | www.shapeblue.com<http://www.shapeblue.com><http://www.shapeblue.com > | ><htp://www.shapeblue.com/> | Twitter:@cloudstackguru > < > https://twitter.com/#!/cloudstackguru> > > ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N > 4HS<x-apple-data-detectors://5> > > > On 7 Apr 2014, at 22:24, "Matthew Midgett" > <supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp > o...@trickhosting.biz>< mailto:supp...@trickhosting.biz>> wrote: > > My system vm's are being created with the wrong routes. Manually > deleting and adding them fixes the problem. Where would I go to fix > this permanently? > > > Sent from my Galaxy S(r)III > Need Enterprise Grade Support for Apache CloudStack? > Our CloudStack Infrastructure Support<http://shapeblue.com/ > cloudstack-infrastructure-support/> offers the best 24/7 SLA for > CloudStack Environments. > > Apache CloudStack Bootcamp training courses > > **NEW!** CloudStack 4.2.1 training<http://shapeblue.com/ > cloudstack-training/> 28th-29th May 2014, Bangalore. > Classromm<http://shapeblue. > com/cloudstack-training/> > 16th-20th June 2014, Region A. Instructor led, On-line< > http://shapeblue.com/cloudstack-training/> > 23rd-27th June 2014, Region B. Instructor led, On-line< > http://shapeblue.com/cloudstack-training/> > 15th-20th September 2014, Region A. Instructor led, On-line< > http://shapeblue.com/cloudstack-training/> > 22nd-27th September 2014, Region B. Instructor led, On-line< > http://shapeblue.com/cloudstack-training/> > 1st-6th December 2014, Region A. Instructor led, On-line< > http://shapeblue.com/cloudstack-training/> > 8th-12th December 2014, Region B. Instructor led, On-line< > http://shapeblue.com/cloudstack-training/> > > This email and any attachments to it may be confidential and are > intended solely for the use of the individual to whom it is addressed. > Any views or opinions expressed are solely those of the author and do > not necessarily represent those of Shape Blue Ltd or related > companies. If you are not the intended recipient of this email, you > must neither take any action based upon its contents, nor copy or show > it to anyone. Please contact the sender if you believe you have > received this email in error. Shape Blue Ltd is a company incorporated > in England & Wales. ShapeBlue Services India LLP is a company > incorporated in India and is operated under license from Shape Blue > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in > Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is > a registered trademark. > > > > > This email and any attachments to it may be confidential and are intended > solely for the use of the individual to whom it is addressed. Any views or > opinions expressed are solely those of the author and do not necessarily > represent those of Shape Blue Ltd or related companies. If you are not the > intended recipient of this email, you must neither take any action based upon > its contents, nor copy or show it to anyone. Please contact the sender if you > believe you have received this email in error. Shape Blue Ltd is a company > incorporated in England & Wales. ShapeBlue Services India LLP is a company > incorporated in India and is operated under license from Shape Blue Ltd. > Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is > operated under license from Shape Blue Ltd. ShapeBlue is a registered > trademark. > > > This email and any attachments to it may be confidential and are intended > solely for the use of the individual to whom it is addressed. Any views or > opinions expressed are solely those of the author and do not necessarily > represent those of Shape Blue Ltd or related companies. If you are not the > intended recipient of this email, you must neither take any action based upon > its contents, nor copy or show it to anyone. Please contact the sender if you > believe you have received this email in error. Shape Blue Ltd is a company > incorporated in England & Wales. ShapeBlue Services India LLP is a company > incorporated in India and is operated under license from Shape Blue Ltd. > Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is > operated under license from Shape Blue Ltd. ShapeBlue is a registered > trademark. This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
