Re: 4.11 without Host-HA framework

[Parth Patel]

Thanks Jon. I've been following that thread too, have now much clear
understanding of how VM HA behaves in cloudstack. Thanks for finding a bug
and saving time of others :)

Regards,
Parth Patel

On Mon 11 Jun, 2018, 16:30 Jon Marshall,  wrote:

> Hi Parth
>
>
> Just in case you have not seen my other thread, it turns out that all this
> time it has been a bug.
>
>
> Using multiple NICs with basic networking and using zone wide NFS VM HA
> just does not work. If you change to cluster wide NFS then it works fine
> (and quite quickly as well :))
>
>
> I am now going to setup Host HA and see make sure that all works as well
> using cluster NFS.
>
>
> Got there in the end :)
>
>
> Jon
>
>
>
>
>
> 
> From: Parth Patel 
> Sent: 24 May 2018 06:52
> To: users@cloudstack.apache.org
> Subject: Re: 4.11 without Host-HA framework
>
> Hi Jon and Angus,
>
> I did not shutdown the VMs as Yiping Zhang said, but I have confirmed this
> and discussed earlier in the users list that my HA-enabled VMs got started
> on another suitable available host in the cluster even when I didn't have
> IPMI-enabled hardware and did no configuration for OOBM and Host-HA. I
> simply pulled the ethernet cable connecting the host to entire network (I
> did use just one NIC) and according to the value set in ping timeout event,
> the HA-enabled VMs were restarted on another available host. I tested the
> scenario using both the scenarios: the echo command as well as good old
> plugging out the NIC from the host. My VMs were successfully started on
> another available host after CS manager confirmed they were not reachable.
>
> I too want to understand how the failover mechanism in CloudStack actually
> works. I used ACS 4.11 packages available here:
> http://cloudstack.apt-get.eu/centos/7/4.11/
>
> Regards,
> Parth Patel
>
>
> On Thu, 24 May 2018 at 10:53 Paul Angus  wrote:
>
> > I'm afraid that is not a host crash.  When shutting down the guest OS,
> the
> > CloudStack agent on the host is still able to report to the management
> > server that the VM has stopped.
> >
> > This is my point. VM-HA relies on the management sever communication with
> > the host agent.
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com<http://www.shapeblue.com>
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Yiping Zhang 
> > Sent: 24 May 2018 00:44
> > To: users@cloudstack.apache.org
> > Subject: Re: 4.11 without Host-HA framework
> >
> > I can say for fact that VM's using a HA enabled service offering will be
> > restarted by CS on another host, assuming there are enough
> > capacity/resources in the cluster, when their original host crashes,
> > regardless that host comes back or not.
> >
> > The simplest way to test VM HA feature with a VM instance using HA
> enabled
> > service offering is to issue shutdown command in guest OS, and watching
> it
> > gets restarted by CS manager.
> >
> > On 5/23/18, 1:23 PM, "Paul Angus"  wrote:
> >
> > Hi Jon,
> >
> > Don't worry, TBH I'm dubious about those claiming to have VM-HA
> > working when a host crashes (but doesn't restart).
> > I'll check in with the guys that set values for host-ha when testing,
> > to see which ones they change and what they set them to.
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com<http://www.shapeblue.com>
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Jon Marshall 
> > Sent: 23 May 2018 21:10
> > To: users@cloudstack.apache.org
> > Subject: Re: 4.11 without Host-HA framework
> >
> > Rohit / Paul
> >
> >
> > Thanks again for answering.
> >
> >
> > I am a Cisco guy with an ex Unix background but no virtualisation
> > experience and I can honestly say I have never felt this stupid before 😊
> >
> >
> > I have Cloudstack working but failover is killing me.
> >
> >
> > When you say VM HA relies on the host telling CS the VM is down how
> > does that work because if you crash the host how does it tell CS
> anything ?
> > And when you say tell CS do you mean the CS manager  ?
> >
> >
> > I guess I am jus

Re: 4.11 without Host-HA framework

[Parth Patel]

Hi Jon and Angus,

I did not shutdown the VMs as Yiping Zhang said, but I have confirmed this
and discussed earlier in the users list that my HA-enabled VMs got started
on another suitable available host in the cluster even when I didn't have
IPMI-enabled hardware and did no configuration for OOBM and Host-HA. I
simply pulled the ethernet cable connecting the host to entire network (I
did use just one NIC) and according to the value set in ping timeout event,
the HA-enabled VMs were restarted on another available host. I tested the
scenario using both the scenarios: the echo command as well as good old
plugging out the NIC from the host. My VMs were successfully started on
another available host after CS manager confirmed they were not reachable.

I too want to understand how the failover mechanism in CloudStack actually
works. I used ACS 4.11 packages available here:
http://cloudstack.apt-get.eu/centos/7/4.11/

Regards,
Parth Patel


On Thu, 24 May 2018 at 10:53 Paul Angus  wrote:

> I'm afraid that is not a host crash.  When shutting down the guest OS, the
> CloudStack agent on the host is still able to report to the management
> server that the VM has stopped.
>
> This is my point. VM-HA relies on the management sever communication with
> the host agent.
>
> Kind regards,
>
> Paul Angus
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Yiping Zhang 
> Sent: 24 May 2018 00:44
> To: users@cloudstack.apache.org
> Subject: Re: 4.11 without Host-HA framework
>
> I can say for fact that VM's using a HA enabled service offering will be
> restarted by CS on another host, assuming there are enough
> capacity/resources in the cluster, when their original host crashes,
> regardless that host comes back or not.
>
> The simplest way to test VM HA feature with a VM instance using HA enabled
> service offering is to issue shutdown command in guest OS, and watching it
> gets restarted by CS manager.
>
> On 5/23/18, 1:23 PM, "Paul Angus"  wrote:
>
> Hi Jon,
>
> Don't worry, TBH I'm dubious about those claiming to have VM-HA
> working when a host crashes (but doesn't restart).
> I'll check in with the guys that set values for host-ha when testing,
> to see which ones they change and what they set them to.
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Jon Marshall 
> Sent: 23 May 2018 21:10
> To: users@cloudstack.apache.org
> Subject: Re: 4.11 without Host-HA framework
>
> Rohit / Paul
>
>
> Thanks again for answering.
>
>
> I am a Cisco guy with an ex Unix background but no virtualisation
> experience and I can honestly say I have never felt this stupid before 😊
>
>
> I have Cloudstack working but failover is killing me.
>
>
> When you say VM HA relies on the host telling CS the VM is down how
> does that work because if you crash the host how does it tell CS anything ?
> And when you say tell CS do you mean the CS manager  ?
>
>
> I guess I am just not understanding all the moving parts. I have had
> HOST HA working (to an extent) although it takes a long time to failover
> even after tweaking the timers but the fact that I keep finding references
> to people saying even without HOST HA it should failover (and mine doesn't)
> makes me think I have configured it incorrectly somewhere along the line.
>
>
> I have configured a compute offering with HA and I am crashing the
> host with the echo command as suggested but still nothing.
>
>
> I understand what you are saying Paul about it not being a good idea
> to rely on VM HA so I will go back to Host HA and try to speed up failover
> times.
>
>
> Can I ask, from your experiences, what is a realistic fail over time
> for CS ie. if a host fails for example ?
>
>
> Jon
>
>
>
>
> 
> From: Paul Angus 
> Sent: 23 May 2018 19:55
> To: users@cloudstack.apache.org
> Subject: RE: 4.11 without Host-HA framework
>
> Jon,
>
> As Rohit says, it is very important to understand the difference
> between VM HA and host HA.
> VM HA relies on the HOST telling CloudStack that the VM is down on
> order for CloudStack start it again (wherever that ends up being).
> Any sequence of events that ends up with VM HA restarting the VM when
> CloudStack can't contact the host is luck/fluke/unreliable/bad(tm)
>
>

Re: CloudStack 4.11 and CCS - shapeblue

[Parth Patel]

Hi Cristian,

I too have tried to test CCS with ACS 4.9 and 4.11 The have been a lot of
architectural changes in ACS version after 4.6 due to which it is a very
challenging to someone who is not a committer to the Cloudstack repository
to modify the source code of CCS that would work with ACS 4.11 and 4.9
(I've tried and failed). I've implemented it with ACS 4.6 and it works
fine. Hope the CloudStack community comes up with a container orchestration
solution like OpenStack's Magnum.

Regards,
Parth Patel

On Thu 19 Apr, 2018, 16:19 Henko Holtzhausen, <
henko.holtzhau...@shapeblue.com> wrote:

> Hi Cristian
>
>
> We are working on supporting CCS with ACS 4.11.
>
> We do not officially support CCS with version 4.9.3.1, the last supported
> version is 4.6.2.1
>
>
> Kind regards
>
> Henko Holtzhausen
>
> 
> From: cristian.c@istream.today 
> Sent: Thursday, April 19, 2018 11:21:46 AM
> To: users@cloudstack.apache.org
> Subject: RE: CloudStack 4.11 and CCS - shapeblue
>
> Update: I also want to know if is compatible with version 4.9.3.1  (
> ShapeBlue packages)
>
>
>
> From: cristian.c@istream.today 
> Sent: Thursday, April 19, 2018 12:12 PM
> To: users@cloudstack.apache.org
> Subject: CloudStack 4.11 and CCS - shapeblue
>
>
>
> Hello Guys,
>
>
>
>  I have a short question, did anyone tested the CCS with the latest
> version of Apache CloudStack 4.11?  ( ShapeBlue repo )
>
>
>
>
>
> Thank you,
>
> Cristian
>
>
> henko.holtzhau...@shapeblue.com
> www.shapeblue.com
> ,
> @shapeblue
>
>
>
>

Re: Untagged Networking for Advanced Zone possible?

[Parth Patel]

Hi Dag,

Thanks a lot  It worked. Now I can mess around and learn how to
configure VPC, autoscale and try my shot at ShapeBlue Container Service
Plugin. Appreciate your help.

Regards,
Parth Patel

On Fri, 6 Apr 2018 at 18:17 Dag Sonstebo  wrote:

> Hi Parth,
>
> No problem, glad I could help. Let us know how you get on with the dummy
> interface bridge.
>
> With regards to your tagging question – yes this is quite a big and
> complicated topic. Suffice to summarise it as follows:
> - Basic zones use a larger L3 network, and guest isolation is done by ACLs
> – in other words firewall rules which controls which VMs can speak to each
> other on the same network.
> - Advanced zones use multiple isolated guest networks which are behind
> virtual routers, and each isolated network must be secured from
> eavesdropping from a neighbour network – to do this you need to put in
> place an isolation mechanism like layer 2 VLAN tags, or a layer 3 SDN
> solution – which uses a slightly different type of tagging to isolate
> networks.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
>
> dag.sonst...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> On 06/04/2018, 13:34, "Parth Patel"  wrote:
>
> Hi Dag,
>
> Thank you for guiding me, i know it's a weird use case and probably
> would
> never be required in a production environment. I will definitely try to
> make a dummy interface and give it to the guest network target bridge.
> I
> know it would be out of the scope of this email trail for you to
> explain me
> tagged and untagged networking in L2 and L3 networks, but I would
> search
> around the internet and ping this thread if I'm again stuck at some
> specific issue after I reach my university's lab. Appreciate your help.
>
> Thanks,
> Parth Patel
>
> On Fri, 6 Apr 2018 at 17:06 Dag Sonstebo 
> wrote:
>
> > Hi Parth,
> >
> > Take a look through the full email trail – I think we discussed this
> > earlier on. In short the answer is no – by definition you can not run
> > completely untagged isolated networks in an advanced zone – but
> “tagged”
> > means different things for L2 and L3 isolation. The real answer - “it
> > depends” – an advanced zone always relies on some sort of guest
> network
> > isolation, which in it’s simplest form equates to L2 VLANs. If you
> were to
> > invest time, effort and money into an SDN solution like Nuage or
> Nicira/NSX
> > you could potentially get around it – but complexity and cost goes
> up. You
> > could have a play with something like GRE tunnelling (L3) – but in my
> > experience this doesn’t scale well, eats a ton of CPU cycles and may
> not be
> > fit for purpose. Again you are looking at a more complex solution.
> >
> > Regarding the dummy network interface it looks to me like a simple
> module
> > install and configuration – see e.g.
> >
> https://www.question-defense.com/2012/11/26/linux-create-fake-ethernet-interface
> > . Not my post and I can’t vouch for it’s validity – but the process
> seems
> > straight forward:
> >
> > [root@kvm1 hooks]# lsmod | grep dummy
> > [root@kvm1 hooks]# modprobe dummy
> > [root@kvm1 hooks]# lsmod | grep dummy
> > dummy   2714  0
> > [root@kvm1 hooks]# ip link set name eth99 dev dummy0
> > [root@kvm1 hooks]# ifconfig eth99
> > eth99 Link encap:Ethernet  HWaddr 92:BF:A6:30:20:3E
> >   BROADCAST NOARP  MTU:1500  Metric:1
> >   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >   TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >   collisions:0 txqueuelen:0
> >   RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
> >
> >
> > Dag Sonstebo
> > Cloud Architect
> > ShapeBlue
> >
> >
> > dag.sonst...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> > On 06/04/2018, 11:45, "Parth Patel" 
> wrote:
> >
> > Hi Dag,
> >
> > Thanks for the response. I am currently looking into VLANs and
> network
> > configuration for my case. But I want to know one thing: are
> "untagged"
> > VLAN networks sufficient for an advanced zone to function with
&

Re: Untagged Networking for Advanced Zone possible?

[Parth Patel]

Hi Dag,

Thank you for guiding me, i know it's a weird use case and probably would
never be required in a production environment. I will definitely try to
make a dummy interface and give it to the guest network target bridge. I
know it would be out of the scope of this email trail for you to explain me
tagged and untagged networking in L2 and L3 networks, but I would search
around the internet and ping this thread if I'm again stuck at some
specific issue after I reach my university's lab. Appreciate your help.

Thanks,
Parth Patel

On Fri, 6 Apr 2018 at 17:06 Dag Sonstebo  wrote:

> Hi Parth,
>
> Take a look through the full email trail – I think we discussed this
> earlier on. In short the answer is no – by definition you can not run
> completely untagged isolated networks in an advanced zone – but “tagged”
> means different things for L2 and L3 isolation. The real answer - “it
> depends” – an advanced zone always relies on some sort of guest network
> isolation, which in it’s simplest form equates to L2 VLANs. If you were to
> invest time, effort and money into an SDN solution like Nuage or Nicira/NSX
> you could potentially get around it – but complexity and cost goes up. You
> could have a play with something like GRE tunnelling (L3) – but in my
> experience this doesn’t scale well, eats a ton of CPU cycles and may not be
> fit for purpose. Again you are looking at a more complex solution.
>
> Regarding the dummy network interface it looks to me like a simple module
> install and configuration – see e.g.
> https://www.question-defense.com/2012/11/26/linux-create-fake-ethernet-interface
> . Not my post and I can’t vouch for it’s validity – but the process seems
> straight forward:
>
> [root@kvm1 hooks]# lsmod | grep dummy
> [root@kvm1 hooks]# modprobe dummy
> [root@kvm1 hooks]# lsmod | grep dummy
> dummy   2714  0
> [root@kvm1 hooks]# ip link set name eth99 dev dummy0
> [root@kvm1 hooks]# ifconfig eth99
> eth99 Link encap:Ethernet  HWaddr 92:BF:A6:30:20:3E
>   BROADCAST NOARP  MTU:1500  Metric:1
>   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>   TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>   collisions:0 txqueuelen:0
>   RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
>
>
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
>
> dag.sonst...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> On 06/04/2018, 11:45, "Parth Patel"  wrote:
>
> Hi Dag,
>
> Thanks for the response. I am currently looking into VLANs and network
> configuration for my case. But I want to know one thing: are "untagged"
> VLAN networks sufficient for an advanced zone to function with two
> networks? I did not state I do not want to use VLANs for networking
> but I
> wanted to know if ACS advanced zone would function if VLANs are
> untagged?
> (The network router/switch did not support them). According to my
> knowledge
> I would have to dig much deeper into kernel files to provide a dummy
> interface as systemctl restart network would not function properly,
> still
> appreciate the advice. I will look into this more and see what I can do
> with it.
>
> Thanks,
> Parth Patel
>
> On Fri, 6 Apr 2018 at 13:59 Dag Sonstebo 
> wrote:
>
> > Hi Parth,
> >
> > Keep in mind you are in unchartered waters – so there may be some
> > stumbling blocks before you get this to work.
> >
> > I suspect what you have to do is change cloudbr1 such that it is
> backed by
> > a fake or dummy ethernet interface. What seems to happen is the agent
> > script looks for the network device backing the bridge rather than
> the
> > bridge itself.
> > This would mean your setup is like this:
> >
> >  Physical eth0 -> cloudbr0 > handles management and public
> >  Dummy eth1 -> cloudbr1 > handles isolated guest traffic and allows
> for
> > isolated VLANs internally on the host
> >
> > Keep in mind the context here – you have stated you don’t want VLANs
>     > traversing your physical network, hence we are trying to get this
> working
> > on a single host only. How you configure your IP ranges for
> management and
> > public is something you need to experiment with and see what works
> for you.
> >
> > Regards,
> > Dag Sonstebo
> > Cloud Architect
> > ShapeBlue
> >
> > From: Parth Patel 
> > Reply-To: "users@cloudstack.apache.org"  >
> &

Re: Untagged Networking for Advanced Zone possible?

[Parth Patel]

Hi Dag,

Thanks for the response. I am currently looking into VLANs and network
configuration for my case. But I want to know one thing: are "untagged"
VLAN networks sufficient for an advanced zone to function with two
networks? I did not state I do not want to use VLANs for networking but I
wanted to know if ACS advanced zone would function if VLANs are untagged?
(The network router/switch did not support them). According to my knowledge
I would have to dig much deeper into kernel files to provide a dummy
interface as systemctl restart network would not function properly, still
appreciate the advice. I will look into this more and see what I can do
with it.

Thanks,
Parth Patel

On Fri, 6 Apr 2018 at 13:59 Dag Sonstebo  wrote:

> Hi Parth,
>
> Keep in mind you are in unchartered waters – so there may be some
> stumbling blocks before you get this to work.
>
> I suspect what you have to do is change cloudbr1 such that it is backed by
> a fake or dummy ethernet interface. What seems to happen is the agent
> script looks for the network device backing the bridge rather than the
> bridge itself.
> This would mean your setup is like this:
>
>  Physical eth0 -> cloudbr0 > handles management and public
>  Dummy eth1 -> cloudbr1 > handles isolated guest traffic and allows for
> isolated VLANs internally on the host
>
> Keep in mind the context here – you have stated you don’t want VLANs
> traversing your physical network, hence we are trying to get this working
> on a single host only. How you configure your IP ranges for management and
> public is something you need to experiment with and see what works for you.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
> From: Parth Patel 
> Reply-To: "users@cloudstack.apache.org" 
> Date: Friday, 6 April 2018 at 07:30
> To: "users@cloudstack.apache.org" 
> Subject: Re: Untagged Networking for Advanced Zone possible?
>
> Hi Dag,
>
> When I tried your method and created a NIC-less bridge, following are the
> contents of my ifcfg-* network files:
>
> ifcfg-cloudbr0:
> TYPE=Bridge
> PROXY_METHOD=none
> BROWSER_ONLY=no
> BOOTPROTO=none
> DEFROUTE=yes
> IPV4_FAILURE_FATAL=no
> NAME=cloudbr0
> UUID=25aabe73-8e11-408f-a4ec-c03b26d3aa6e
> DEVICE=cloudbr0
> ONBOOT=yes
> IPADDR=172.16.20.13
> PREFIX=16
> GATEWAY=172.16.0.1
> DNS1=8.8.8.8
> DNS2=172.16.0.1
> NM_CONTROLLED=no
>
> ifcfg-cloudbr1:
> TYPE=Bridge
> IPV4_FAILURE_FATAL=no
> NAME=cloudbr1
> UUID=25aabe73-8e11-408f-a4ec-c03b26d3aa6e
> DEVICE=cloudbr1
> ONBOOT=yes
> NM_CONTROLLED=no
>
> ifcfg-eno1:
> TYPE=Ethernet
> PROXY_METHOD=none
> BROWSER_ONLY=no
> BOOTPROTO=none
> DEFROUTE=yes
> IPV4_FAILURE_FATAL=no
> NAME=eno1
> UUID=25aabe73-8e11-408f-a4ec-c03b26d3aa6e
> DEVICE=eno1
> ONBOOT=yes
> IPADDR=172.16.20.13
> PREFIX=16
> GATEWAY=172.16.0.1
> DNS1=8.8.8.8
> DNS2=172.16.0.1
> NM_CONTROLLED=no
> BRIDGE=cloudbr0
>
> brctl show output:
> [root@srvr3 ~]# brctl show
> bridge name bridge id STP enabled interfaces
> cloud0 8000. no
> cloudbr0 8000.3464a92a09f3 no eno1
> cloudbr1 8000. no
> virbr0 8000.5254002dabdb yes virbr0-nic
>
>
>
> when adding a host in advanced zone it shows the following error: Could
> not find network 'cloudbr1'
>
> 2018-04-04 02:03:11,887 DEBUG [c.c.u.s.SSHCmdHelper]
> (qtp510113906-14:ctx-707b53e5 ctx-8d49ccb3) (logid:dff92f23) Executing cmd:
> /usr/share/cloudstack-common/scripts/util/keystore-cert-import
> /etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/
> 2018-04-04 02:03:15,686 DEBUG [c.c.h.k.d.LibvirtServerDiscoverer]
> (qtp510113906-14:ctx-707b53e5 ctx-8d49ccb3) (logid:dff92f23) Succeeded to
> import certificate in the keystore for agent on the KVM host: 172.16.20.13.
> Agent secured and trusted.
> 2018-04-04 02:03:15,688 DEBUG [c.c.u.s.SSHCmdHelper]
> (qtp510113906-14:ctx-707b53e5 ctx-8d49ccb3) (logid:dff92f23) Executing cmd:
> cloudstack-setup-agent  -m 172.16.20.13 -z 1 -p 1 -c 1 -g
> 1fd67886-c5d9-3464-ac73-46689258b34e -a --pubNic=cloudbr0 --prvNic=cloudbr0
> --guestNic=cloudbr1 --hypervisor=kvm
> 2018-04-04 02:03:19,674 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-af4b26a6) (logid:4c5c40d4) Begin cleanup
> expired async-jobs
> 2018-04-04 02:03:19,683 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-af4b26a6) (logid:4c5c40d4) End cleanup expired
> async-jobs
> 2018-04-04 02:03:20,022 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-f1d46df0) (logid:a021b44c) Found 0 routers to
> update status.
> 2018-04-04 02:03:20,025 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]

Re: Untagged Networking for Advanced Zone possible?

[Parth Patel]

Impl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.hypervisor.xenserver.discoverer.XcpServerDiscoverer
2018-04-04 02:04:24,002 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.hypervisor.hyperv.discoverer.HypervServerDiscoverer
2018-04-04 02:04:24,002 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.vm.ClusteredVirtualMachineManagerImpl
2018-04-04 02:04:24,002 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.network.security.SecurityGroupListener
2018-04-04 02:04:24,002 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: org.apache.cloudstack.engine.orchestration.NetworkOrchestrator
2018-04-04 02:04:24,002 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.storage.secondary.SecondaryStorageListener
2018-04-04 02:04:24,002 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.storage.listener.StoragePoolMonitor
2018-04-04 02:04:24,002 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.deploy.DeploymentPlanningManagerImpl
2018-04-04 02:04:24,002 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.storage.LocalStoragePoolListener
2018-04-04 02:04:24,002 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.network.SshKeysDistriMonitor
2018-04-04 02:04:24,002 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl
2018-04-04 02:04:24,002 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.consoleproxy.ConsoleProxyListener
2018-04-04 02:04:24,005 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.network.SshKeysDistriMonitor
2018-04-04 02:04:24,005 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.network.router.VirtualNetworkApplianceManagerImpl
2018-04-04 02:04:24,005 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener:
com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
2018-04-04 02:04:24,005 DEBUG [c.c.n.NetworkUsageManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Disconnected called
on 1 with status Alert
2018-04-04 02:04:24,006 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.storage.download.DownloadListener
2018-04-04 02:04:24,006 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.capacity.StorageCapacityListener
2018-04-04 02:04:24,006 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.capacity.ComputeCapacityListener
2018-04-04 02:04:24,006 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.agent.manager.AgentManagerImpl$BehindOnPingListener
2018-04-04 02:04:24,006 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.agent.manager.AgentManagerImpl$SetHostParamsListener
2018-04-04 02:04:24,006 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Sending Disconnect
to listener: com.cloud.storage.upload.UploadListener
2018-04-04 02:04:24,007 DEBUG [c.c.h.Status]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Transition:[Resource
state = Enabled, Agent event = AgentDisconnected, Host id = 1, name =
srvr3.cloud.priv]
2018-04-04 02:04:24,028 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Notifying other
nodes of to disconnect
2018-04-04 02:04:24,034 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Failed to handle
host connection: com.cloud.exception.ConnectionException: Incorrect Network
setup on agent, Reinitialize agent after network names are setup, details :
Can not find network: cloudbr1
2018-04-04 02:04:24,046 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-1:ctx-0bf7b058) (logid:f7a4d646) Can no

Re: Not able to import template from internal network

[Parth Patel]

Soundar,

I too have faced the same issue in ACS 4.11. The default port (80) won't
work. For me, changing the httpd server's port to 8080 worked. I think it's
a bug in the iptables or ingress rules of SSVM. I too am able to wget or
curl from inside of SSVM. Can anyone please explain this behaviour?

Regards,
Parth Patel

On Tue 3 Apr, 2018, 17:17 Swastik Mittal,  wrote:

> Soundar
>
> Do you have your sshd services available on your internal network.
>
> On 3 Apr 2018 3:47 p.m., "soundar rajan"  wrote:
>
> > Further debugging found the below errors
> >
> > INFO [Storage.template.httptemplatedownloader] No credentials configured
> > for host=172.xx.xx.xx:80
> > Starting download from http://172.xx.xx.xx/SVR.qcow2  to
> > /mnt/SecStorage/0d869-xx-xx-xxx-xx/template/2/245/dndlxxtmp_
> > remotesize-27839829 max size-32232323
> > Download completion for job id xxx  status=unrecoverable_error
> >
> > The above log is fron SSVM
> >
> > Regards
> > Shyam
> >
> > On Tue, Apr 3, 2018 at 3:29 PM, soundar rajan 
> > wrote:
> >
> > > i logged in to the SSVM from management console login in using root and
> > > able to download the QCow2 file using wget
> > >
> > > @ Swastik
> > >
> > > The error message it says as stream closed (its a qcow2 format not an
> > iso)
> > >
> > > On Tue, Apr 3, 2018 at 3:21 PM, Dag Sonstebo <
> dag.sonst...@shapeblue.com
> > >
> > > wrote:
> > >
> > >> Soundar,
> > >>
> > >> The first step in troubleshooting is to simply log on to the SSVM,
> then
> > >> try to do a wget against the internal URL.
> > >>
> > >> Regards,
> > >> Dag Sonstebo
> > >> Cloud Architect
> > >> ShapeBlue
> > >>
> > >> On 03/04/2018, 10:49, "soundar rajan"  wrote:
> > >>
> > >> But the same works from the template downloaded from internet  for
> > >> eg..
> > >>
> > >> http://cloud.centos.org/Centos7.xx.qcow2 ( Works fine)
> > >> http://172.xx.xx.xx/SVR.qcow2 (says stream closed not working)
> > >>
> > >> secstorage.allowed.internal.sites: 172.xx.0.0/16
> > >>
> > >> On Tue, Apr 3, 2018 at 3:14 PM, soundar rajan <
> > bsoundara...@gmail.com
> > >> >
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > I am not able to import template from internal network. I haved
> > >> added the
> > >> > ip in the global config and configured the same.
> > >> >
> > >> > Please find the error message
> > >> >
> > >> > 2018-04-03 15:12:29,323 DEBUG [c.c.a.t.Request]
> > >> > (AgentManager-Handler-7:null) (logid:) Seq
> 2-4749608757015609350:
> > >> > Processing:  { Ans: , MgmtId: 2887018115, via: 2, Ver: v1,
> Flags:
> > >> 10,
> > >> > [{"com.cloud.agent.api.storage.DownloadAnswer":{"
> > >> > jobId":"37d6c445-25b2-4d0c-9c5e-264152ff76bc","
> > >> > downloadPct":0,"errorString":"Stream closed","downloadStatus":"
> > >> > DOWNLOAD_ERROR","downloadPath":"/mnt/SecStorage/0d869cc4-
> > >> > 6763-391b-83ef-4b686f2ac7a2/template/tmpl/2/243/
> > >> > dnld7748993734774922951tmp_","installPath":"template/tmpl/2/
> > >> > 243","templateSize":0,"templatePhySicalSize":0,"
> > >> > checkSum":"","result":true,"details":"Stream closed","wait":0}}]
> > }
> > >> >
> > >> > Regards
> > >> > Shyam
> > >> >
> > >>
> > >>
> > >>
> > >> dag.sonst...@shapeblue.com
> > >> www.shapeblue.com
> > >> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > >> @shapeblue
> > >>
> > >>
> > >>
> > >>
> > >
> >
>

Re: Untagged Networking for Advanced Zone possible?

[Parth Patel]

Thanks Dag. Appreciate it. Will try this out.

On Thu, 29 Mar 2018 at 16:02 Dag Sonstebo 
wrote:

> Hi Parth,
>
> If you want a KVM networking introduction take a look at my blog post from
> a couple of years back – this is still valid:
> http://www.shapeblue.com/networking-kvm-for-cloudstack/
>
> In short – you don’t set up VLAN tagging for isolated networks on the KVM
> host – you set up the bridge and then specify your VLAN range when you set
> up your zone in CloudStack. CloudStack then takes care of creating the
> isolated VLAN isolated networks on the host. So in short – you create your
> bridges, then use the bridge names in the advanced zone setup.
>
> Virtual bridge – yes this is similar to the cloud0 bridge, and yes you
> create the bridge without a physical interface.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
>
> dag.sonst...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> On 29/03/2018, 11:14, "Parth Patel"  wrote:
>
> Hi Dag,
>
> Thanks for helping me understand the requirement of advanced
> networking.
> Sorry if I have missed something obvious or my question seems stupid,
> but I
> am just starting to learn. Can you help me out on how to setup VLAN
> "tagging" on one machine? I have tried several methods and tutorials I
> could find on the internet for VLANs, but none mention "tagging".
>
> Also, I do not fully understand private virtual bridge. Means I
> create
> an interface file for bridge but mention no physical interface device?
> Is
> it similar to how cloud0 is configured for link local network of System
> VMs? I could probably do that, but I don't know much about configuring
> VLAN
> tagging. I would appreciate if you could give me some guidance or
> point me
> where you think some good documentation is given for CentOS/RHEL hosts
> for
> configuring bridges with VLAN tagging (I have tried but failed to
> understand most of them). I am especially stuck at understanding this
> "tagging" of VLANs.
>
> Thanks,
> Parth Patel
>
> On Thu, 29 Mar 2018 at 15:17 Dag Sonstebo 
> wrote:
>
> > Hi Parth,
> >
> > Yes and no.
> >
> > No – you cannot do advanced zones with *all three* KVM hosts and
> advanced
> > networking without using VLANs (or another isolation mechanism) and
> still
> > expect traffic to flow between VMs/VRs on different KVM hosts.
> >
> > Yes – you can probably do this *on a single KVM host* – but you will
> have
> > to use VLAN tagging internally – this can however be done on a
> virtual
> > bridge interface, i.e. the L2 traffic doesn’t ever leave that host.
> >
> > Without deep diving into this I think it would look like this:
> >
> > Physical eth0 -> cloudbr0 > handles management and public
> > No nic -> private virtual bridge cloudbr1 > handles isolated guest
> traffic
> > but allows for isolated VLANs internally on the host
> >
> > Regards,
> > Dag Sonstebo
> > Cloud Architect
> > ShapeBlue
> >
> >
> > dag.sonst...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> > On 29/03/2018, 09:25, "Parth Patel" 
> wrote:
> >
> > Hi Dag,
> >
> > Thanks for the reply. I am trying to use Shapeblue CCS
> (Container as a
> > Service) with ACS, but for that Isolated networks are required
> which
> > are
> > only available in Advanced Zone. Further, I want to explore
> Cloudstack
> > further and am also aiming to test and configure other advanced
> > features
> > such as load balancing and auto scaling without netscaler
> device. For
> > that
> > I badly need Advanced Zone networking (especially isolated
> networks
> > offerings). I just want to know if Advanced Zone can succesfully
> > function
> > with two networks, one physcial NIC and no VLAN tagging.
> >
> > Thanks,
> > Parth Patel
> >
> > On Thu, 29 Mar 2018 at 13:48 Dag Sonstebo <
> dag.sonst...@shapeblue.com>
> > wrote:
> >
> > > Hi Parth,
> > >
> > > Not sure if I follow. Generally, 

Re: Untagged Networking for Advanced Zone possible?

[Parth Patel]

Hi Dag,

Thanks for helping me understand the requirement of advanced networking.
Sorry if I have missed something obvious or my question seems stupid, but I
am just starting to learn. Can you help me out on how to setup VLAN
"tagging" on one machine? I have tried several methods and tutorials I
could find on the internet for VLANs, but none mention "tagging".

Also, I do not fully understand private virtual bridge. Means I create
an interface file for bridge but mention no physical interface device? Is
it similar to how cloud0 is configured for link local network of System
VMs? I could probably do that, but I don't know much about configuring VLAN
tagging. I would appreciate if you could give me some guidance or point me
where you think some good documentation is given for CentOS/RHEL hosts for
configuring bridges with VLAN tagging (I have tried but failed to
understand most of them). I am especially stuck at understanding this
"tagging" of VLANs.

Thanks,
Parth Patel

On Thu, 29 Mar 2018 at 15:17 Dag Sonstebo 
wrote:

> Hi Parth,
>
> Yes and no.
>
> No – you cannot do advanced zones with *all three* KVM hosts and advanced
> networking without using VLANs (or another isolation mechanism) and still
> expect traffic to flow between VMs/VRs on different KVM hosts.
>
> Yes – you can probably do this *on a single KVM host* – but you will have
> to use VLAN tagging internally – this can however be done on a virtual
> bridge interface, i.e. the L2 traffic doesn’t ever leave that host.
>
> Without deep diving into this I think it would look like this:
>
> Physical eth0 -> cloudbr0 > handles management and public
> No nic -> private virtual bridge cloudbr1 > handles isolated guest traffic
> but allows for isolated VLANs internally on the host
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
>
> dag.sonst...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> On 29/03/2018, 09:25, "Parth Patel"  wrote:
>
> Hi Dag,
>
> Thanks for the reply. I am trying to use Shapeblue CCS (Container as a
> Service) with ACS, but for that Isolated networks are required which
> are
> only available in Advanced Zone. Further, I want to explore Cloudstack
> further and am also aiming to test and configure other advanced
> features
> such as load balancing and auto scaling without netscaler device. For
> that
> I badly need Advanced Zone networking (especially isolated networks
> offerings). I just want to know if Advanced Zone can succesfully
> function
> with two networks, one physcial NIC and no VLAN tagging.
>
> Thanks,
> Parth Patel
>
> On Thu, 29 Mar 2018 at 13:48 Dag Sonstebo 
> wrote:
>
> > Hi Parth,
> >
> > Not sure if I follow. Generally, your management network is untagged,
> > whilst your public and isolated networks tagged. The underlying idea
> of
> > advanced zones is you must have network isolation between multiple
> guest
> > networks, otherwise you have no privacy/security. You can do this
> either at
> > L2 with VLAN tagging, which is the simplest, or with L3 using
> various SDN
> > overlay network solutions (more complicated and comes at a cost).
> >
> > If you don’t want to tag anything you’re probably better off using
> basic
> > networks, where I believe you could use a single flat subnet (happy
> to be
> > proven wrong).
> >
> > Regards,
> > Dag Sonstebo
> > Cloud Architect
> > ShapeBlue
> >
> >
> > dag.sonst...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> > On 29/03/2018, 08:48, "Parth Patel" 
> wrote:
> >
> > Hi all,
> >
> > After banging my head with different network configuration
> > permutations, I
> > don't understand what is the issue with Network Guru here and
> why it
> > can't
> > implement the isolated guest network. I just want to know if
> Advanced
> > Zone
> > can be successfully setup or has someone configured an advanced
> zone
> > using
> > untagged VLAN traffic?
> >
> > I have the following configuration of components:
> > - I have 3 (16 GB Ram and 4 Cores) machines each with 1 physical
> NIC.
> > - I have two networks: 192.168.20.0/24 (using this for isolated
> g

Re: Untagged Networking for Advanced Zone possible?

[Parth Patel]

Hi Dag,

Thanks for the reply. I am trying to use Shapeblue CCS (Container as a
Service) with ACS, but for that Isolated networks are required which are
only available in Advanced Zone. Further, I want to explore Cloudstack
further and am also aiming to test and configure other advanced features
such as load balancing and auto scaling without netscaler device. For that
I badly need Advanced Zone networking (especially isolated networks
offerings). I just want to know if Advanced Zone can succesfully function
with two networks, one physcial NIC and no VLAN tagging.

Thanks,
Parth Patel

On Thu, 29 Mar 2018 at 13:48 Dag Sonstebo 
wrote:

> Hi Parth,
>
> Not sure if I follow. Generally, your management network is untagged,
> whilst your public and isolated networks tagged. The underlying idea of
> advanced zones is you must have network isolation between multiple guest
> networks, otherwise you have no privacy/security. You can do this either at
> L2 with VLAN tagging, which is the simplest, or with L3 using various SDN
> overlay network solutions (more complicated and comes at a cost).
>
> If you don’t want to tag anything you’re probably better off using basic
> networks, where I believe you could use a single flat subnet (happy to be
> proven wrong).
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
>
> dag.sonst...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> On 29/03/2018, 08:48, "Parth Patel"  wrote:
>
> Hi all,
>
> After banging my head with different network configuration
> permutations, I
> don't understand what is the issue with Network Guru here and why it
> can't
> implement the isolated guest network. I just want to know if Advanced
> Zone
> can be successfully setup or has someone configured an advanced zone
> using
> untagged VLAN traffic?
>
> I have the following configuration of components:
> - I have 3 (16 GB Ram and 4 Cores) machines each with 1 physical NIC.
> - I have two networks: 192.168.20.0/24 (using this for isolated guest
> network) and 172.16.20.0/16 (management server and NFS servers
> network)
> - I am using KVM hypervisor and NFS for storage.
> - Currently, the output of brctl show is (when the Cloudstack is not
> running, other wise the interface are populated with three vnets for
> cloud0
> and 4-5 vnets for cloudbr0):
> bridge name bridge id   STP enabled interfaces
> cloud0  8000.   no
> cloudbr08000.3464a92a083a   no  eno1
> virbr0  8000.525400daae23   yes virbr0-nic
>
> My earlier doubt was if I can configure advanced zone with one physical
> interface available in each host, but that was resolved when I read
> this
> post of ShankerBalan:
>
> https://shankerbalan.net/blog/cloudstack-simple-advanced-network-example/
>
> ACS throws InsufficientVirtualNetworkCapacity exception and lines like:
> "NetworkGuru can't implement network [275||15]" are printed in
> management
> server logs when I try to create a simple CentOS 5.5 NoGUI KVM instance
> after a complete and fresh install of ACS (even of CentOS).
>
> My main doubt here is if I can successfully configure an advanced zone
> with
> two networks but with untagged VLAN traffic ? I can't currently
> configure
> the router or switches to allow tagged VLAN networking as I am doing
> this
> project in my university. But, I have requested and gained access to
> the
> mentioned two networks: 192.168.20.0/24 and 172.16.20.0/16 and both
> networks are pingable and have internet access across all three
> machines.
> Can anyone help me with this please?
>
> Thanks,
> Parth Patel
>
>
>

Untagged Networking for Advanced Zone possible?

[Parth Patel]

Hi all,

After banging my head with different network configuration permutations, I
don't understand what is the issue with Network Guru here and why it can't
implement the isolated guest network. I just want to know if Advanced Zone
can be successfully setup or has someone configured an advanced zone using
untagged VLAN traffic?

I have the following configuration of components:
- I have 3 (16 GB Ram and 4 Cores) machines each with 1 physical NIC.
- I have two networks: 192.168.20.0/24 (using this for isolated guest
network) and 172.16.20.0/16 (management server and NFS servers network)
- I am using KVM hypervisor and NFS for storage.
- Currently, the output of brctl show is (when the Cloudstack is not
running, other wise the interface are populated with three vnets for cloud0
and 4-5 vnets for cloudbr0):
bridge name bridge id   STP enabled interfaces
cloud0  8000.   no
cloudbr08000.3464a92a083a   no  eno1
virbr0  8000.525400daae23   yes virbr0-nic

My earlier doubt was if I can configure advanced zone with one physical
interface available in each host, but that was resolved when I read this
post of ShankerBalan:
https://shankerbalan.net/blog/cloudstack-simple-advanced-network-example/

ACS throws InsufficientVirtualNetworkCapacity exception and lines like:
"NetworkGuru can't implement network [275||15]" are printed in management
server logs when I try to create a simple CentOS 5.5 NoGUI KVM instance
after a complete and fresh install of ACS (even of CentOS).

My main doubt here is if I can successfully configure an advanced zone with
two networks but with untagged VLAN traffic ? I can't currently configure
the router or switches to allow tagged VLAN networking as I am doing this
project in my university. But, I have requested and gained access to the
mentioned two networks: 192.168.20.0/24 and 172.16.20.0/16 and both
networks are pingable and have internet access across all three machines.
Can anyone help me with this please?

Thanks,
Parth Patel

Re: [VOTE] Move to Github issues

[Parth Patel]

+1

On Mon, 26 Mar 2018 at 21:25 Glenn Wagner 
wrote:

> +1
>
> glenn.wag...@shapeblue.com
> www.shapeblue.com
> Winter Suite, 1st Floor, The Avenues, Drama Street, Somerset West, Cape
> Town  7129South Africa
> @shapeblue
>
>
>
>
> -Original Message-
> From: David Mabry 
> Sent: Monday, 26 March 2018 4:51 PM
> To: d...@cloudstack.apache.org
> Cc: users 
> Subject: Re: [VOTE] Move to Github issues
>
> +1
>
> On 3/26/18, 8:05 AM, "Will Stevens"  wrote:
>
> +1
>
> On Mon, Mar 26, 2018, 5:51 AM Nicolas Vazquez, <
> nicolas.vazq...@shapeblue.com> wrote:
>
> > +1
> >
> > 
> > From: Dag Sonstebo 
> > Sent: Monday, March 26, 2018 5:06:29 AM
> > To: users@cloudstack.apache.org; d...@cloudstack.apache.org
> > Subject: Re: [VOTE] Move to Github issues
> >
> > +1
> >
> > Regards,
> > Dag Sonstebo
> > Cloud Architect
> > ShapeBlue
> >
> > On 26/03/2018, 07:33, "Rohit Yadav"  wrote:
> >
> > All,
> >
> > Based on the discussion last week [1], I would like to start a
> vote to
> > put
> > the proposal into effect:
> >
> > - Enable Github issues, wiki features in CloudStack repositories.
> > - Both user and developers can use Github issues for tracking
> issues.
> > - Developers can use #id references while fixing an existing/open
> > issue in
> > a PR [2]. PRs can be sent without requiring to open/create an
> issue.
> > - Use Github milestone to track both issues and pull requests
> towards a
> > CloudStack release, and generate release notes.
> > - Relax requirement for JIRA IDs, JIRA still to be used for
> historical
> > reference and security issues. Use of JIRA will be discouraged.
> > - The current requirement of two(+) non-author LGTMs will
> continue for
> > PR
> > acceptance. The two(+) PR non-authors can advise resolution to
> any
> > issue
> > that we've not already discussed/agreed upon.
> >
> > For sanity in tallying the vote, can PMC members please be sure
> to
> > indicate
> > "(binding)" with their vote?
> >
> > [ ] +1  approve
> > [ ] +0  no opinion
> > [ ] -1  disapprove (and reason why)
> >
> > Vote will be open for 120 hours. If the vote passes the following
> > actions
> > will be taken:
> > - Get Github features enabled from ASF INFRA
> > - Update CONTRIBUTING.md and other relevant cwik
> i
> pages.
> > - Update project website
> >
> > [1] https://markmail.org/message/llodbwsmzgx5hod6
> > [2]
> > https://blog.github.com/2013-05-14-closing-issues-via-pull-requests/
> >
> > Regards,
> > Rohit Yadav
> >
> >
> >
> > dag.sonst...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > nicolas.vazq...@shapeblue.com
> > www.shapeblue.com
> > ,
> > @shapeblue
> >
> >
> >
> >
>
>
>

Re: Container Service Requirements

[Parth Patel]

Hi Paul,

Currently, I am unable to create a normal VM instance of CentOS 5.5 No GUI
KVM in my advanced zone. I am able to create isolated guest networks in the
advanced zone but when creating a VM, ACS throws the same error -
InsufficientVirtualNetworkCapacity and NetworkGuru was not able to
implement Network [225||17]...

Do I need a separate physical NIC or manually need to create a separate
bridge for the same NIC for the guest network? (I don't know if it is
possible to create two bridges using different networks using one physical
NIC)

Thanks,
Parth Patel

On Fri, 23 Mar 2018 at 13:49 Paul Angus  wrote:

> Parth,
>
> CCS sits on top of CloudStack so is agnostic of the physical
> configuration, as long as your isolated networking is ok, then CCS will be
> able to function on top of them.
>
> Sounds like you need to troubleshoot your isolated networks, try
> independently creating isolated networks to ensure you can create them ok.
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Parth Patel 
> Sent: 22 March 2018 20:33
> To: users@cloudstack.apache.org
> Subject: Container Service Requirements
>
> Hi all,
>
> I need someone to guide me through Shapeblue ccs service configuration. I
> am currently using ACS 4.6 with it. I have following components available
> at my exposure:
>
> - I have 3 (16 GB Ram and 4 Cores) machines each with 1 physical NIC.
> - I have two networks: 192.168.20.1/24 (using this for isolated guest
> network) and 172.16.20.0/16 (management server and NFS servers network)
> - I am using KVM hypervisor and NFS for storage.
> - Currently, the output of brctl show is:
> bridge name bridge id   STP enabled interfaces
> cloud0  8000.   no
> cloudbr08000.3464a92a083a   no  eno1
> virbr0  8000.525400daae23   yes virbr0-nic
>
> First I didn't know that CCS requires Advanced Zone to work (uses Isolated
> Guest Network), but still after having two different network CIDRs, when I
> try to create a container cluster with just 1 node (cluster size), ACS
> throws InsufficientVirtualNetworkCapacity exception and lines like:
> "NetworkGuru can't implement network [275||15]" are printed in management
> server logs.
>
> My main doubt is are two physical separate NICs required to make this work?
> It would be beneficial if anyone who has successfully configured Shapeblue
> CCS service shares their implementation details or tell me if I am missing
> something.
>
> Thanks,
> Parth Patel
>

Container Service Requirements

[Parth Patel]

Hi all,

I need someone to guide me through Shapeblue ccs service configuration. I
am currently using ACS 4.6 with it. I have following components available
at my exposure:

- I have 3 (16 GB Ram and 4 Cores) machines each with 1 physical NIC.
- I have two networks: 192.168.20.1/24 (using this for isolated guest
network) and 172.16.20.0/16 (management server and NFS servers network)
- I am using KVM hypervisor and NFS for storage.
- Currently, the output of brctl show is:
bridge name bridge id   STP enabled interfaces
cloud0  8000.   no
cloudbr08000.3464a92a083a   no  eno1
virbr0  8000.525400daae23   yes virbr0-nic

First I didn't know that CCS requires Advanced Zone to work (uses Isolated
Guest Network), but still after having two different network CIDRs, when I
try to create a container cluster with just 1 node (cluster size), ACS
throws InsufficientVirtualNetworkCapacity exception and lines like:
"NetworkGuru can't implement network [275||15]" are printed in management
server logs.

My main doubt is are two physical separate NICs required to make this work?
It would be beneficial if anyone who has successfully configured Shapeblue
CCS service shares their implementation details or tell me if I am missing
something.

Thanks,
Parth Patel

Re: KVM HostHA

[Parth Patel]

Hi Jon,

I have to admit that I have a beginner/mediocre understanding of cloudstack
overall (especially the host HA feature). But what works for me should work
for everyone. So, to answer your questions:

1) how many compute nodes do you have
>

I have tested using three agents as when using only two nodes, management
server deemed one node which was running system VMs and router as unfit for
migration and stopped the VM. I currently use one node for execution of
system VMs and router, and two agents (compute nodes you can say) out of
which one is running a  HA-enabled VM and one agent running 0 VMs running
as I only have 4GB ram in each of those :| I use one machine (fourth one)
for running management server and MySQL database. I also have the 5th
machine separate purely for NFS. Although, you can easily have management,
MySQL and NFS setup on the same machine (depends on your machine's
configuration/capacity)

>
>
> 2) are you running basic or advanced networking
>

I am using basic (flat) networking where my management IP addresses range
from 172
16.4.131 to 172.16.4.137 and guest IP addresses range from 172.16.4.138 to
172.16.4.149. Both are on a /24 network.

>
> 3) how have you setup your NICs ie. on each compute node I have 3 separate
> NICs, one for management, one for the VMs and one for storage (NFS).
>

I only have 1 NIC per machine (same is used for all 3 types of traffic). I
have seen management server use peer routing from other agents to perform
some operations in my XenCluster but I highly doubt this would be the case
your management server does not mark a host as "Down" (as I said I don't
know about internal working of Cloudstack but just a guess as I've seen in
management server logs) I suggest you remove all three NICs of a host for
simulating my scenario.

>
>
> So far I have not managed to get any failover of VMs no matter what I try
>

I also recommend you update your qemu-kvm, NFS and other packages (there
has just been a recent update for CentOS 7) (again I know this is
superstitious but still, sometimes different package versions have been
known to be the root cause of the issue)

Side note: my ACS 4.11 agent auto reboots itself after it has retried
communicating with management server 4 times, at almost the exact same time
management server decides in its logs that the host and HA-enabled VM has
stopped executing and it restarts the HA-enabled VM on another host.


Hope this helps.

Regards,
Parth Patel.

>
>
> 
> From: Parth Patel 
> Sent: 14 March 2018 14:36
> To: users@cloudstack.apache.org
> Subject: Re: KVM HostHA
>
> Hi Paul and Adrina,
>
> I don't know the functioning of Host-HA features but what Paul explained,
> my ACS 4.11 does the same without even host HA or ipmi access. As I stated
> earlier multiple times, without host HA and ipmi, my ha-enabled VMs
> executing on a normal host get restarted on another suitable host in
> cluster after approximately 3 minutes of event ping timeout. After which
> the cloudstack agent with no connection to management server because of
> unplugged NIC (all my machines currently have only one NIC / whole zone is
> in a flat network) reboots itself (the reason was explained by Rohit in an
> another thread). The management server marks the host down and only
> Ha-enabled VMs executing on it get restarted on another host (without any
> mention of host HA or ipmi or fencing in management server logs) while
> normal VMs executing on it are stopped.
>
> I don't know if this was a desired outcome, but I think my current ACS 4.11
> installation has features (at least performs some ;) provided by Host HA
> without configuring it or ipmi.
>
> Regards,
> Parth Patel
>
> On Wed 14 Mar, 2018, 18:41 Boris Stoyanov, 
> wrote:
>
> > yes, KVM + NFS shared storage.
> >
> > Boris.
> >
> >
> > boris.stoya...@shapeblue.com
> > www.shapeblue.com<http://www.shapeblue.com>
> [http://www.shapeblue.com/wp-content/uploads/2017/06/logo.png]<
> http://www.shapeblue.com/>
>
> Shapeblue - The CloudStack Company<http://www.shapeblue.com/>
> www.shapeblue.com
> Rapid deployment framework for Apache CloudStack IaaS Clouds. CSForge is a
> framework developed by ShapeBlue to deliver the rapid deployment of a
> standardised ...
>
>
>
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> > > On 14 Mar 2018, at 14:51, Andrija Panic 
> wrote:
> > >
> > > Hi Boris,
> > >
> > > ok thanks for the explanation - that makes sense, and covers my
> > "exception
> > > case" that I have.
> > >
> > > This is atm only available fo

Re: KVM HostHA

[Parth Patel]

Hi Paul and Adrina,

I don't know the functioning of Host-HA features but what Paul explained,
my ACS 4.11 does the same without even host HA or ipmi access. As I stated
earlier multiple times, without host HA and ipmi, my ha-enabled VMs
executing on a normal host get restarted on another suitable host in
cluster after approximately 3 minutes of event ping timeout. After which
the cloudstack agent with no connection to management server because of
unplugged NIC (all my machines currently have only one NIC / whole zone is
in a flat network) reboots itself (the reason was explained by Rohit in an
another thread). The management server marks the host down and only
Ha-enabled VMs executing on it get restarted on another host (without any
mention of host HA or ipmi or fencing in management server logs) while
normal VMs executing on it are stopped.

I don't know if this was a desired outcome, but I think my current ACS 4.11
installation has features (at least performs some ;) provided by Host HA
without configuring it or ipmi.

Regards,
Parth Patel

On Wed 14 Mar, 2018, 18:41 Boris Stoyanov, 
wrote:

> yes, KVM + NFS shared storage.
>
> Boris.
>
>
> boris.stoya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> > On 14 Mar 2018, at 14:51, Andrija Panic  wrote:
> >
> > Hi Boris,
> >
> > ok thanks for the explanation - that makes sense, and covers my
> "exception
> > case" that I have.
> >
> > This is atm only available for NFS as I could read (KVM on NFS) ?
> >
> > Cheers
> >
> > On 14 March 2018 at 13:02, Boris Stoyanov 
> > wrote:
> >
> >> Hi Andrija,
> >>
> >> There’s two types of checks Host-HA is doing to determine if host if
> >> healthy.
> >>
> >> 1. Health checks - pings the host as soon as there’s connection issues
> >> with the agent
> >>
> >> If that fails,
> >>
> >> 2. Activity checks - checks if there are any writing operations on the
> >> Disks of the VMs that are running on the hosts. This is to determine if
> the
> >> VMs are actually alive and executing processes. Only if no disk
> operations
> >> are executed on the shared storage, only then it’s trying to Recover the
> >> host with IPMI call, if that eventually fails, it migrates the VMs to a
> >> healthy host and Fences the faulty one.
> >>
> >> Hope that explains your case.
> >>
> >> Boris.
> >>
> >>
> >> boris.stoya...@shapeblue.com
> >> www.shapeblue.com
> >> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> >> @shapeblue
> >>
> >>
> >>
> >>> On 14 Mar 2018, at 13:53, Andrija Panic 
> wrote:
> >>>
> >>> Hi Paul,
> >>>
> >>> sorry to bump in the middle of the thread, but just curious about the
> >> idea
> >>> behing host-HA and why it behaves the way you exlained above:
> >>>
> >>>
> >>> Would it be more sense (or not?), that when MGMT detects agents is
> >>> unreachable or host unreachable (or after unsuccessful i.e. agent
> >> restart,
> >>> etc...,to be defined), to actually use IPMI to STONITH the node, thus
> >>> making sure no VMS running and then to really start all HA-enabled VMs
> on
> >>> other hosts ?
> >>>
> >>> I'm just trying to make parallel to the corosync/pacemaker as
> clustering
> >>> suite/services in Linux (RHEL and others), where when majority of nodes
> >>> detect that one node is down, a common thing (especially for shared
> >>> storage) is to STONITH that node, make sure it;s down, then move
> >> "resource"
> >>> (in our case VMs) to other cluster nodes ?
> >>>
> >>> I see it's  actually much broader setup per
> >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Host+HA but
> >> again -
> >>> whole idea (in my head at least...) is when host get's down, we make
> sure
> >>> it's down (avoid VM corruption, by doint STONITH to that node) and then
> >>> start HA VMs on ohter hosts.
> >>>
> >>> I understand there might be exceptions as I have right now (4.8) -
> >> libvirt
> >>> get stuck (librbd exception or similar) so agent get's disconnected,
> but
> >>> VMs are still running fine... (except DB get messed up, all NICs loose
> >>> isolation_uri, VR's loose MAC addresses and other IP addresses 

Re: KVM HostHA

[Parth Patel]

Hi Paul,

Thanks for the clarification. I currently don't have an ipmi enabled
hardware (in test environment), but it will be beneficial if you can help
me clear out some basic concepts of it:
- If HA-enabled VMs are autostarted on another host when current host goes
down, what is the need or purpose of HA-host? (other than management server
able to remotely control it's power interfaces)
- I understood the "Shoot-the-other-node-in-the-head" (STONITH) approach
ACS uses to fence the host, but I couldn't find what mechanism or events
trigger this?

Thanks and regards,
Parth Patel

On Wed, 14 Mar 2018 at 02:22 Paul Angus  wrote:

> The management server doesn't ping the host through IPMI.   However if
> IPMI is not available, you will not be able to use Host HA, as there is no
> way for CloudStack to 'fence' the host - that is shut it down to be sure
> that a VM cannot start again on that host.
>
> I can explain why that is necessary if you wish.
>
>
> Kind regards,
>
> Paul Angus
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Parth Patel 
> Sent: 13 March 2018 16:57
> To: users@cloudstack.apache.org
> Cc: Jon Marshall 
> Subject: Re: KVM HostHA
>
> Hi Jon and Victor,
>
> I think the management server pings your host using ipmi (I really don't
> hope this is the case).
> In my case, I did not have OOBM enabled at all (my hardware didn't support
> it)
> I think you could disable OOBM and/or HA-Host and give that a try :)
>
> On Tue, 13 Mar 2018 at 20:40 victor  wrote:
>
> > Hello Guys,
> >
> > I have tried the following two cases.
> >
> > 1, "echo c > /proc/sysrq-trigger"
> >
> > 2, Pulled the network cable of one of the host
> >
> > In both cases, the following happened.
> >
> > =
> > 2018-03-13 08:22:54,978 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> > (AgentTaskPool-15:ctx-c8d9f5d2) (logid:c0a3d2da) Notifying other nodes
> > of to disconnect
> > 2018-03-13 08:22:54,983 INFO [c.c.a.m.AgentManagerImpl]
> > (AgentTaskPool-16:ctx-d8204625) (logid:ffe4a426) Host 4 is
> > disconnecting with event AgentDisconnected
> > 2018-03-13 08:22:54,985 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentTaskPool-16:ctx-d8204625) (logid:ffe4a426) Host 4 is already
> > Alert
> > 2018-03-13 08:22:54,985 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentTaskPool-16:ctx-d8204625) (logid:ffe4a426) Deregistering link
> > for
> > 4 with state Alert
> > 2018-03-13 08:22:54,985 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentTaskPool-16:ctx-d8204625) (logid:ffe4a426) Remove Agent : 4
> > =
> >
> > But nothing happened for the  vm's in that node. I have waited for one
> > hour and the VM's in that node has been migrated to the other
> > available hosts. I think the issue is that the management server still
> > thinks that the VM's in that host is running. Please check the
> > following logs
> >
> > ===
> > 2018-03-13 11:08:25,882 DEBUG [c.c.c.CapacityManagerImpl]
> > (CapacityChecker:ctx-1d8378af) (logid:ae906a50) Found 1 VMs on host 4
> > 2018-03-13 11:08:25,888 DEBUG [c.c.c.CapacityManagerImpl]
> > (CapacityChecker:ctx-1d8378af) (logid:ae906a50) Found 0 VM, not
> > running on host 4 
> >
> >
> > On 03/13/2018 04:20 PM, Jon Marshall wrote:
> > > I tried "echo c > /proc/sysrq-trigger" which stopped me getting into
> > > the
> > server but it did not stop the server responding to an ipmitool
> > request on the manager eg -
> > >
> > >
> > > "ipmitool -I lanplus -H 172.16.7.29 -U admin3 -P letmein chassis
> status"
> > >
> > >
> > > from the management server got an answer saying the chassis power
> > > was on
> > so CS never registered the compute node as down.
> > >
> > >
> > > I am obviously doing something wrong but cannot work it out.
> > >
> > >
> > > The management server has one NIC - 172.16.7.4
> > >
> > >
> > > Each compute node has 3 NICs -
> > >
> > >
> > > cnode1
> > cnode2
> > >
> > >
> > > mangement NIC172.16.7.5   172.16.7.6
> > >
> > > vm NIC  172.16.6.130 172.16.6.131
> > >
> > > storage - 172.16.250.4

Re: CCS (Cloudstack Container Service) in CloudStack 4.11

[Parth Patel]

Oh, ok. Thanks for the info Daan.

On Tue 13 Mar, 2018, 23:32 Daan Hoogland,  wrote:

> Don't bother. Ccs will not work with 4.9
>
> Biligual auto correct use.  Read at your own risico
>
> On 13 Mar 2018 18:05, "Parth Patel"  wrote:
>
> Hi Da'an,
>
> I will give that a try, till then I will try the ccs first with ACS 4.9 :P
>
> Thanks,
> Parth Patel
>
> On Tue, 13 Mar 2018 at 17:39 Daan Hoogland 
> wrote:
>
> > Parth, The plugin 1.0 won't work with any version beyond 4.9 (4.7 and 4.7
> > aren't tested). There is a branch, 'on-top-of-pr-2071-for-4.10
> > <https://github.com/shapeblue/ccs/tree/on-top-of-pr-2071-for-4.10>',
> that
> > we are now porting to 4.11. If you have capability to build it you can
> try
> > that.
> >
> > On Tue, Mar 13, 2018 at 1:03 PM, Parth Patel 
> > wrote:
> >
> > > Hi,
> > >
> > > Maybe somebody working in ShapeBlue can help me. I am trying to
> integrate
> > > CCS service (https://github.com/shapeblue/ccs) with ACS 4.11 using
> > > http://downloads.shapeblue.com/ccs/1.0/Installation_and_
> > > Administration_Guide.pdf
> > > and
> > > I quickly came to know about huge architecture changes that ACS has
> gone
> > > through in the latest release. I somehow managed to bring the UI by
> > editing
> > > and placing the ccs folder in the new "webapp" folder instead of
> > "webapps"
> > > folder. But I am confused as to where to place the shapeblue-ccs and
> > > flyaway core jar files. I tried placing them in
> > > /usr/share/cloudstack-management/lib folder but it gave the following
> > > error
> > > in management-server.log:
> > >
> > >
> > > 2018-03-13 16:59:27,953 INFO
> [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > > (main:null) (logid:) Loading module context [ccs] from URL
> > > [jar:file:/usr/share/cloudstack-management/webapp/
> > > WEB-INF/lib/cloud-plugin-shapeblue-ccs-1.0.1.jar!/META-
> > > INF/cloudstack/ccs/spring-ccs-context.xml]
> > > 2018-03-13 16:59:27,953 INFO
> [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > > (main:null) (logid:) Loading module context [ccs] from URL
> > > [jar:file:/usr/share/cloudstack-management/lib/
> > > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/compute/spring-
> > > core-lifecycle-compute-context-inheritable.xml]
> > > 2018-03-13 16:59:27,953 INFO
> [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > > (main:null) (logid:) Loading module context [ccs] from URL
> > > [jar:file:/usr/share/cloudstack-management/lib/
> > > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/core/spring-
> > > core-lifecycle-core-context-inheritable.xml]
> > > 2018-03-13 16:59:27,953 INFO
> [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > > (main:null) (logid:) Loading module context [ccs] from URL
> > > [jar:file:/usr/share/cloudstack-management/lib/
> > > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/system/spring-
> > > framework-config-system-context-inheritable.xml]
> > > 2018-03-13 16:59:27,953 INFO
> [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > > (main:null) (logid:) Loading module context [ccs] from URL
> > > [jar:file:/usr/share/cloudstack-management/lib/
> > > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/system/spring-
> > > core-system-context-inheritable.xml]
> > > 2018-03-13 16:59:27,953 INFO
> [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > > (main:null) (logid:) Loading module context [ccs] from URL
> > > [jar:file:/usr/share/cloudstack-management/lib/
> > > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/system/spring-
> > > contrail-system-context-inheritable.xml]
> > > 2018-03-13 16:59:27,953 INFO
> [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > > (main:null) (logid:) Loading module context [ccs] from URL
> > > [jar:file:/usr/share/cloudstack-management/lib/
> > > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/bootstrap/
> > > spring-bootstrap-context-inheritable.xml]
> > > 2018-03-13 16:59:28,098 WARN  [o.a.c.s.m.c.ResourceApplicationContext]
> > > (main:null) (logid:) Exception encountered during context
> initialization
> > -
> > > cancelling refresh attempt:
> > > org.springframework.beans.factory.BeanCreationException: Error creating
> > > bean with name 'ContainerClusterManagerImpl': Lookup method resolution
> > > failed; nested exception is java.lang.IllegalStateException: Failed to
> > > introspect Cla

Re: CCS (Cloudstack Container Service) in CloudStack 4.11

[Parth Patel]

Hi Da'an,

I will give that a try, till then I will try the ccs first with ACS 4.9 :P

Thanks,
Parth Patel

On Tue, 13 Mar 2018 at 17:39 Daan Hoogland  wrote:

> Parth, The plugin 1.0 won't work with any version beyond 4.9 (4.7 and 4.7
> aren't tested). There is a branch, 'on-top-of-pr-2071-for-4.10
> <https://github.com/shapeblue/ccs/tree/on-top-of-pr-2071-for-4.10>', that
> we are now porting to 4.11. If you have capability to build it you can try
> that.
>
> On Tue, Mar 13, 2018 at 1:03 PM, Parth Patel 
> wrote:
>
> > Hi,
> >
> > Maybe somebody working in ShapeBlue can help me. I am trying to integrate
> > CCS service (https://github.com/shapeblue/ccs) with ACS 4.11 using
> > http://downloads.shapeblue.com/ccs/1.0/Installation_and_
> > Administration_Guide.pdf
> > and
> > I quickly came to know about huge architecture changes that ACS has gone
> > through in the latest release. I somehow managed to bring the UI by
> editing
> > and placing the ccs folder in the new "webapp" folder instead of
> "webapps"
> > folder. But I am confused as to where to place the shapeblue-ccs and
> > flyaway core jar files. I tried placing them in
> > /usr/share/cloudstack-management/lib folder but it gave the following
> > error
> > in management-server.log:
> >
> >
> > 2018-03-13 16:59:27,953 INFO  [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > (main:null) (logid:) Loading module context [ccs] from URL
> > [jar:file:/usr/share/cloudstack-management/webapp/
> > WEB-INF/lib/cloud-plugin-shapeblue-ccs-1.0.1.jar!/META-
> > INF/cloudstack/ccs/spring-ccs-context.xml]
> > 2018-03-13 16:59:27,953 INFO  [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > (main:null) (logid:) Loading module context [ccs] from URL
> > [jar:file:/usr/share/cloudstack-management/lib/
> > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/compute/spring-
> > core-lifecycle-compute-context-inheritable.xml]
> > 2018-03-13 16:59:27,953 INFO  [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > (main:null) (logid:) Loading module context [ccs] from URL
> > [jar:file:/usr/share/cloudstack-management/lib/
> > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/core/spring-
> > core-lifecycle-core-context-inheritable.xml]
> > 2018-03-13 16:59:27,953 INFO  [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > (main:null) (logid:) Loading module context [ccs] from URL
> > [jar:file:/usr/share/cloudstack-management/lib/
> > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/system/spring-
> > framework-config-system-context-inheritable.xml]
> > 2018-03-13 16:59:27,953 INFO  [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > (main:null) (logid:) Loading module context [ccs] from URL
> > [jar:file:/usr/share/cloudstack-management/lib/
> > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/system/spring-
> > core-system-context-inheritable.xml]
> > 2018-03-13 16:59:27,953 INFO  [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > (main:null) (logid:) Loading module context [ccs] from URL
> > [jar:file:/usr/share/cloudstack-management/lib/
> > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/system/spring-
> > contrail-system-context-inheritable.xml]
> > 2018-03-13 16:59:27,953 INFO  [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
> > (main:null) (logid:) Loading module context [ccs] from URL
> > [jar:file:/usr/share/cloudstack-management/lib/
> > cloudstack-4.11.0.0.jar!/META-INF/cloudstack/bootstrap/
> > spring-bootstrap-context-inheritable.xml]
> > 2018-03-13 16:59:28,098 WARN  [o.a.c.s.m.c.ResourceApplicationContext]
> > (main:null) (logid:) Exception encountered during context initialization
> -
> > cancelling refresh attempt:
> > org.springframework.beans.factory.BeanCreationException: Error creating
> > bean with name 'ContainerClusterManagerImpl': Lookup method resolution
> > failed; nested exception is java.lang.IllegalStateException: Failed to
> > introspect Class [com.cloud.containercluster.ContainerClusterManagerImpl]
> > from ClassLoader [WebAppClassLoader=500772834@1dd92fe2]
> > 2018-03-13 16:59:28,100 WARN  [o.e.j.w.WebAppContext] (main:null)
> (logid:)
> > Failed startup of context o.e.j.w.WebAppContext@365c30cc
> > {/client,file:///usr/share/cloudstack-management/webapp/,
> > UNAVAILABLE}{/usr/share/cloudstack-management/webapp}
> > org.springframework.beans.factory.BeanCreationException: Error creating
> > bean with name 'ContainerClusterManagerImpl': Lookup method resolution
> > failed; nested exception is java.lang.IllegalStateException: Failed to
&

Re: KVM HostHA

[Parth Patel]

Hi Jon and Victor,

I think the management server pings your host using ipmi (I really don't
hope this is the case).
In my case, I did not have OOBM enabled at all (my hardware didn't support
it)
I think you could disable OOBM and/or HA-Host and give that a try :)

On Tue, 13 Mar 2018 at 20:40 victor  wrote:

> Hello Guys,
>
> I have tried the following two cases.
>
> 1, "echo c > /proc/sysrq-trigger"
>
> 2, Pulled the network cable of one of the host
>
> In both cases, the following happened.
>
> =
> 2018-03-13 08:22:54,978 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentTaskPool-15:ctx-c8d9f5d2) (logid:c0a3d2da) Notifying other nodes
> of to disconnect
> 2018-03-13 08:22:54,983 INFO [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-16:ctx-d8204625) (logid:ffe4a426) Host 4 is disconnecting
> with event AgentDisconnected
> 2018-03-13 08:22:54,985 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-16:ctx-d8204625) (logid:ffe4a426) Host 4 is already Alert
> 2018-03-13 08:22:54,985 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-16:ctx-d8204625) (logid:ffe4a426) Deregistering link for
> 4 with state Alert
> 2018-03-13 08:22:54,985 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-16:ctx-d8204625) (logid:ffe4a426) Remove Agent : 4
> =
>
> But nothing happened for the  vm's in that node. I have waited for one
> hour and the VM's in that node has been migrated to the other available
> hosts. I think the issue is that the management server still thinks that
> the VM's in that host is running. Please check the following logs
>
> ===
> 2018-03-13 11:08:25,882 DEBUG [c.c.c.CapacityManagerImpl]
> (CapacityChecker:ctx-1d8378af) (logid:ae906a50) Found 1 VMs on host 4
> 2018-03-13 11:08:25,888 DEBUG [c.c.c.CapacityManagerImpl]
> (CapacityChecker:ctx-1d8378af) (logid:ae906a50) Found 0 VM, not running
> on host 4
> 
>
>
> On 03/13/2018 04:20 PM, Jon Marshall wrote:
> > I tried "echo c > /proc/sysrq-trigger" which stopped me getting into the
> server but it did not stop the server responding to an ipmitool request on
> the manager eg -
> >
> >
> > "ipmitool -I lanplus -H 172.16.7.29 -U admin3 -P letmein chassis status"
> >
> >
> > from the management server got an answer saying the chassis power was on
> so CS never registered the compute node as down.
> >
> >
> > I am obviously doing something wrong but cannot work it out.
> >
> >
> > The management server has one NIC - 172.16.7.4
> >
> >
> > Each compute node has 3 NICs -
> >
> >
> > cnode1
> cnode2
> >
> >
> > mangement NIC172.16.7.5   172.16.7.6
> >
> > vm NIC  172.16.6.130 172.16.6.131
> >
> > storage - 172.16.250.4   172.16.250.5
> >
> >
> > Dell LOM (for Idrac)   172.16.7.29172.16.7.30
> >
> >
> > the dell LOM IPs are the ones used to configure OOBM  in the UI
> >
> >
> >
> > If I pull the storage NIC presumably nothing will happen as the ipmitool
> check is running across the management NIC so I need to pull both ?
> >
> > My understanding of host HA was the management server monitored the
> compute nodes using ipmitool and if it did not get a response because the
> host was down it would fence off that host and move the VMs to an active
> compute node.
> >
> > This is obviously too simplistic so could someone explain how it is
> meant to work and what it is protecting against ?
> >
> > 
> > From: Paul Angus 
> > Sent: 13 March 2018 07:01
> > To: users@cloudstack.apache.org
> > Subject: RE: KVM HostHA
> >
> > Hi all,
> >
> > One small note, unplugging the management NIC will only cause an HA
> event if the storage is running over that NIC also.
> >
> > Is the storage is over a separate NIC then, the guest VMs will continue
> to run when the mgmt. NIC is unplugged, Host HA will detect the disk
> activity and conclude that there is nothing it can do, as the VMs are still
> running other than mark the hosts as degraded.
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com<http://www.shapeblue.com>
> > [http://www.shapeblue.com/wp-content/uploads/2017/06/logo.png]<
> http://www.shapeblue.com/>
> >
> > Shapeblue - The CloudStack Company<http://www.shapeblue.com/>
> > www.shapeblue.com
> > Rapid deployment fr

CCS (Cloudstack Container Service) in CloudStack 4.11

[Parth Patel]

inerClusterManagerImpl] from ClassLoader
[WebAppClassLoader=500772834@1dd92fe2]
at
org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:659)
at
org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:556)
at
org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:541)
... 59 more
Caused by: java.lang.NoClassDefFoundError:
org/bouncycastle/asn1/DEREncodable
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.getDeclaredMethods(Class.java:1975)
... 62 more
Caused by: java.lang.ClassNotFoundException:
org.bouncycastle.asn1.DEREncodable
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
... 68 more
2018-03-13 16:59:28,106 INFO  [o.e.j.s.h.ContextHandler] (main:null)
(logid:) Started o.e.j.s.h.MovedContextHandler@80169cf{/,null,AVAILABLE}
2018-03-13 16:59:28,125 INFO  [o.e.j.s.AbstractConnector] (main:null)
(logid:) Started ServerConnector@15d0c81b{HTTP/1.1,[http/1.1]}{:::8080}
2018-03-13 16:59:28,125 INFO  [o.e.j.s.Server] (main:null) (logid:) Started
@30618ms



I also tried placing them in
/usr/share/cloudstack-management/lib/cloudstack-4.11.0.0.jar but it showed
the following error:

Caused by: java.io.IOException: Resource
[jar:file:/usr/share/cloudstack-management/lib/cloud-plugin-shapeblue-ccs-1.0.1.jar!/META-INF/cloudstack/ccs/module.properties]
and
[jar:file:/usr/share/cloudstack-management/webapp/WEB-INF/lib/cloud-plugin-shapeblue-ccs-1.0.1.jar!/META-INF/cloudstack/ccs/module.properties]
do not appear to be the same resource, please ensure the name property is
correct or that the module is not defined twice

I suppose I would have to change module.properties or context files of the
ccs code? I don't know where to start. Even the write your plugin guide
here:http://docs.cloudstack.apache.org/en/latest/plugins.html guides
writing UI plugins for tomcat and other storage services. Can somebody
point me to an updated guide for writing plugins *now* in ACS 4.11? so that
I can verify and modify if required the GitHub code available at
shapeblue/ccs. Any guidance is appreciated.

Thanks,
Parth Patel

Re: KVM HostHA

[Parth Patel]

>
> Hi Jon,
>
> As I said, in my case, making the host HA didn't work but by just having a
> HA VM running on host and executing - (WARNING) "echo c >
> /proc/sysrq-trigger" to simulate a kernel crash on host, the management
> server registered it as down and started the VM on another host. I know
> I've suggested this before but I insist you give this a try. Also, you
> don't need to completely power off the machine manually but just plugging
> out the network cable works fine. The cloudstack agent after losing
> connection to management server auto reboots because of KVM heartbeat check
> shell script mentioned by Rohit Yadav to one of my earlier queries in other
> thread.
>
> On Mon 12 Mar, 2018, 21:23 Jon Marshall,  wrote:
> Hi Paul
>
>
> Thanks for the response.
>
>
> I think I am not understanding how it was meant to work then. My
> understanding was that the manager used ipmitool to just keep querying the
> compute nodes as to their status so I assumed it didn't matter how you shut
> the node down, once it was down the manager would get no response and mark
> it as down (which it does).
>
>
> I am in testing mode so I think I will just go and pull the power and see
> what happens :)
>
>
> Thanks
>
>
> Jon
>
>
> 
> From: Paul Angus 
> Sent: 12 March 2018 15:31
> To: users@cloudstack.apache.org
> Subject: RE: KVM HostHA
> Hi Jon,
>
> I think that what you guys are finding, is that a controlled host
> shutdown, which will cause the agent to shutdown cleanly; Is not considered
> an HA event. I wouldn't expect CloudStack to take any action if you shut
> down a host, only if the host (agent) stops responding.
>
>
>
>
> Kind regards,
>
> Paul Angus
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> [http://www.shapeblue.com/wp-content/uploads/2017/06/logo.png]<
> http://www.shapeblue.com/>
>
> Shapeblue - The CloudStack Company
> www.shapeblue.com
> Rapid deployment framework for Apache CloudStack IaaS Clouds. CSForge is a
> framework developed by ShapeBlue to deliver the rapid deployment of a
> standardised ...
>
>
>
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Jon Marshall 
> Sent: 12 March 2018 15:15
> To: users@cloudstack.apache.org
> Subject: Re: KVM HostHA
>
> I have the same issue here and am not entirely sure what the behaviour
> should be.
>
>
> I have one manager node and 2 compute nodes running 4.11 with ipmi working
> correctly.
>
>
> From the UI under HA -
>
>
> HA Enabled Yes
> HA State Available
> HA Provider kvmhaprovider
>
>
> although interestingly from the "Details" tab it shows -
>
>
> HA enabled No
>
>
> which I assume is a cosmetic issue ?
>
>
> On each compute node I have one HA enabled VM and one non HA enabled VM.
>
>
> I power off a compute node and the UI updates the host status and the VMs
> on that node stop responding but they never fail over to the other node.
>
>
> Couple of things I noticed -
>
>
> 1) as soon as i power off the compute node the HA state on the other node
> shows "Ineligible"
>
>
> 2) In the UI the instances all still show as green even though two of them
> are not available
>
>
> Any help much appreciated
>
>
>
>
> 
> From: victor 
> Sent: 07 March 2018 17:01
> To: users@cloudstack.apache.org
> Subject: KVM HostHA
>
> Hello Guys,
>
> I have installed cloudstack 4.11. I have enabled HA for each hosts I have
> added. I have also added ipmi successfully (using ipmi driver).
> The hosts are showing like the following.
>
> ===
>
> HA Enabled Yes
> HA State Available
> HA Provider kvmhaprovider
>
> ==
>
> Also the host is showing the following correctly
>
> Resource state --> Enabled
> State --> UP
> Power state --> On
>
> So I have shutdown one of the hosts to see how the KVM hosts Ha is
> working. I have waited for half an hour. But nothing has happened. What
> will happen to the VM's in that host, if the host failed to back up.
> There isn't much from logs.
>
> Regards
> Victor
>

Re: KVM HostHA

[Parth Patel]

Hi Victor,

I too had a similar failover requirement. I also got on the path of making
an HA-enabled KVM host in CS 4.9 after doing the same steps you performed
but in CS 4.9 where the agent got in "Alert" state but not in "Down" state.
However, if your requirement is simply that in case a host executing an
HA-enabled VM goes down in CS 4.11, it should be restarted on another host,
you don't need to make the KVM host HA.

How I replicated failover scenario in Cloudstack 4.11:
- Start an HA-enabled VM on a host.
- Unplug the host
- Make sure at least one suitable host with enough resources is available.
- My CS 4.11 after the durations of ping-duration*ping-timeout (60*2.5 ~
3.5 minutes) decides that the host is down and restarts the VM on another
host.
(NOTE: this assumes that your NFS or storage server is on another machine
and you are not using local storage for the HA-enabled VM)

Your management server logs should show that host id: xxx has disconnected
with event ping timeout and after several of those messages, it should
decide that host is down. If this is not the case, look for insufficient
server capacity and cannot create deployment messages in server logs. If
all of above don't match your scenario, a look at management server logs
would help.

On Thu, 8 Mar 2018 at 14:05 victor  wrote:

> Hello Andrija,
>
> Yes I am doing the same test as you mentioned ie unplug NIC in one of
> the host and observer the action of VM's in that host. But in my test
> the VM's didn't get started in another host.
>
> Regards
> Victor
>
>
> On 03/07/2018 11:52 PM, Andrija Panic wrote:
> > Hi Victor,
> >
> > zero experience here with 4.11 in general, but what are you expecting to
> > happen ?
> >
> > you powered off a host, so nothing for IPMI driver to do - host is down
> > already, no host HA actions are expected afaik.
> >
> > I guess you might have have wanted to i.e. unplug NIC (cause network
> issues
> > on MGMT network), or... kill agent service and then observe the actions.
> >
> > Were VMs started on another host, in your test?
> >
> > Cheers
> >
> > On 7 March 2018 at 18:01, victor  wrote:
> >
> >> Hello Guys,
> >>
> >> I have installed cloudstack 4.11. I have enabled HA for each hosts I
> have
> >> added. I have also added ipmi successfully (using ipmi driver).   The
> hosts
> >> are showing like the following.
> >>
> >> ===
> >>
> >> HA Enabled  Yes
> >> HA StateAvailable
> >> HA Provider kvmhaprovider
> >>
> >> ==
> >>
> >> Also the host is showing the following correctly
> >>
> >> Resource state --> Enabled
> >> State --> UP
> >> Power state --> On
> >>
> >> So I have shutdown one of the hosts to see how the KVM hosts Ha is
> >> working.  I have waited for half an hour. But nothing has happened. What
> >> will happen to the VM's in that host, if the host failed to back up.
> There
> >> isn't much from logs.
> >>
> >> Regards
> >> Victor
> >>
> >
> >
>
>

Re: Cloud (Infrastructure reselling) = VPC ?

[Parth Patel]

Thank you very much Andrija, I was scraping through cloudstack
administration docs to properly understand this. I'll certainly ask for
your guidance if I get stuck somewhere. Thank you again :)

On Mon 5 Mar, 2018, 19:03 Andrija Panic,  wrote:

> Doman architecture setup:
>
> Domain: /PARTNER1/client1/
> Domain:  /PARTNER1/client2/...
>
> Domain:  /PARTNER2/client1/
> Domain:  /PARTNER2/client2/ ...
>
> partner=reseler, while you keep to be provider of IaaS for your reselers...
> This all above is if you leave the "selling" to you "partners"
>
> In each PARTNERX domain, you will make one DOMAIN ADMIN account for your
> partner/resseler to be able to administer it's user's VMs/resources
> Inside each /client1/ or /client2/ subdomains (subdomains inside partner's
> domains) you would create regular user accounts (with optionally multiple
> users for such account)
>
> If you have more specific question, I might be able to help (we have such
> setup in place)
>
> You could build central user (web) portal, where users can apply for
> registration, than when an Portal Admin approves, client's would be
> provisioned from Portal via API/root keys on CloudStack side, or similar.
> (you need some mechanism of applying for account on CloudStack - and later
> for user to be able to see some details, or to change their passwords - via
> User portal for example, etc)
>
> Then, white-labeling with custom CSS and logos, etc... :)
>
> Cheers
>
> On 4 March 2018 at 10:33, Parth Patel  wrote:
>
> > Thank you DAG and Ivan. I will go ahead with this.
> >
> > On Sun 4 Mar, 2018, 14:08 Ivan Kudryavtsev, 
> > wrote:
> >
> > > I suppose sometimes It's wise to use Domains rather than accounts, but
> If
> > > you have single-rank customers (every customer fits the same rules)
> then
> > > accounts are just fine.
> > >
> > > 2018-03-04 15:35 GMT+07:00 Dag Sonstebo :
> > >
> > > > Hi Parth
> > > >
> > > > To maintain current and future client isolation you should look at
> > > setting
> > > > up an account (not just a user) for each client, then in each account
> > you
> > > > can configure a VPC or simple isolated network for the webservers.
> > > >
> > > > Hope this helps,
> > > > Dag Sonstebo
> > > >
> > > > Get Outlook for iOS<https://aka.ms/o0ukef>
> > > > 
> > > > From: Parth Patel 
> > > > Sent: Saturday, March 3, 2018 9:45:27 PM
> > > > To: users@cloudstack.apache.org
> > > > Subject: Cloud (Infrastructure reselling) = VPC ?
> > > >
> > > > Hi,
> > > >
> > > > I just recently got a bit experience in CloudStack. My company wants
> to
> > > > host web servers for clients who from their panel or administration
> > > > dashboard have only basic privileges with the servers/VMs that are
> > > running.
> > > > For this reselling type of requirement, I started to dig into
> Projects
> > > view
> > > > and user permission control but did not find much. From my
> > understanding
> > > > what I need to create is a VPC (Virtual Private Cloud) offering for
> > each
> > > > client. right ? can anybody guide me o
> <https://maps.google.com/?q=%0A%3E+%3E+%3E+client.+right+?+can+anybody+guide+me+o&entry=gmail&source=g>n
> this, I just want to know I am
> > > > heading in right direction.
> > > >
> > > > Regards,
> > > > Parth Patel
> > > >
> > > > dag.sonst...@shapeblue.com
> > > > www.shapeblue.com
> > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > > @shapeblue
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > With best regards, Ivan Kudryavtsev
> > > Bitworks Software, Ltd.
> > > Cell: +7-923-414-1515
> > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > >
> >
>
>
>
> --
>
> Andrija Panić
>

Re: Cloud (Infrastructure reselling) = VPC ?

[Parth Patel]

Thank you DAG and Ivan. I will go ahead with this.

On Sun 4 Mar, 2018, 14:08 Ivan Kudryavtsev, 
wrote:

> I suppose sometimes It's wise to use Domains rather than accounts, but If
> you have single-rank customers (every customer fits the same rules) then
> accounts are just fine.
>
> 2018-03-04 15:35 GMT+07:00 Dag Sonstebo :
>
> > Hi Parth
> >
> > To maintain current and future client isolation you should look at
> setting
> > up an account (not just a user) for each client, then in each account you
> > can configure a VPC or simple isolated network for the webservers.
> >
> > Hope this helps,
> > Dag Sonstebo
> >
> > Get Outlook for iOS<https://aka.ms/o0ukef>
> > 
> > From: Parth Patel 
> > Sent: Saturday, March 3, 2018 9:45:27 PM
> > To: users@cloudstack.apache.org
> > Subject: Cloud (Infrastructure reselling) = VPC ?
> >
> > Hi,
> >
> > I just recently got a bit experience in CloudStack. My company wants to
> > host web servers for clients who from their panel or administration
> > dashboard have only basic privileges with the servers/VMs that are
> running.
> > For this reselling type of requirement, I started to dig into Projects
> view
> > and user permission control but did not find much. From my understanding
> > what I need to create is a VPC (Virtual Private Cloud) offering for each
> > client. right ? can anybody guide me on this, I just want to know I am
> > heading in right direction.
> >
> > Regards,
> > Parth Patel
> >
> > dag.sonst...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
>
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ <http://bw-sw.com/>
>

Cloud (Infrastructure reselling) = VPC ?

[Parth Patel]

Hi,

I just recently got a bit experience in CloudStack. My company wants to
host web servers for clients who from their panel or administration
dashboard have only basic privileges with the servers/VMs that are running.
For this reselling type of requirement, I started to dig into Projects view
and user permission control but did not find much. From my understanding
what I need to create is a VPC (Virtual Private Cloud) offering for each
client. right ? can anybody guide me on this, I just want to know I am
heading in right direction.

Regards,
Parth Patel

Re: HA-enabled VM not starting or migrating to another host if current host goes down

[Parth Patel]

Hi Sean,

Thank you for the reply. I was already testing Cloudstack and now tried the
same with 4.11. It works as it should in Cloudstack 4.11. However, I think
that pull request should work and will test it out with remaining 3
machines with Cloudstack 4.9.

Regards,
Parth Patel

On Fri, 2 Mar 2018 at 03:42 Sean Lair  wrote:

> Yea, we had the exact same problem.  VM HA is broken in 4.9 - at least
> under KVM
>
> We use this Pull Request in our environment to fix it.
>
> https://github.com/apache/cloudstack/pull/2474
>
> However, as stated in the PR, enable libvirt (edit /etc/libvirt/qemu.conf
> and change and comment: lock_manager = "lockd")
>
> https://libvirt.org/locking-lockd.html
>
>
>
> -Original Message-
> From: Parth Patel [mailto:parthpatel2...@gmail.com]
> Sent: Saturday, February 24, 2018 10:54 PM
> To: users@cloudstack.apache.org
> Subject: Re: HA-enabled VM not starting or migrating to another host if
> current host goes down
>
> Hi Simon,
>
> I'm using KVM hypervisor.
>
> On Sun 25 Feb, 2018, 01:41 Simon Weller,  wrote:
>
> > Which hypervisor are you using?
> >
> > Simon Weller/615-312-6068 <(615)%20312-6068>
> >
> > -Original Message-
> > From: Parth Patel [parthpatel2...@gmail.com]
> > Received: Saturday, 24 Feb 2018, 11:33AM
> > To: users@cloudstack.apache.org [users@cloudstack.apache.org]
> > Subject: HA-enabled VM not starting or migrating to another host if
> > current host goes down
> >
> > Hi,
> >
> > I am developing an enterprise-level cloud infrastructure currently
> > using Cloudstack 4.9. My head of department wished to check a specific
> > failsafe scenario. It is as follows:
> >
> > A highly available VM whose continuous execution even in event of some
> > storage or connection error, should remain running or migrate to
> > another host automatically if anything were to happen to the current
> > host. I tried removing the LAN cable from the current host after an
> > HA-enabled VM was executing on it, but the management server would not
> > auto-start the VM on another host. It kept printing error messages
> > such as "Communication failure. Host 5 timed out due to even
> > PingTimeout" in management server logs. I have manually set the ping
> > timeout duration to 30 seconds and its multiplier value to 1.
> >
> > Any suggestions as to what extra configuration is needed to make
> > Cloudstack start the VM on another host or migrate it?
> >
> > Just to be clear, the host on which the HA-enabled VM is running does
> > not have its primary and secondary storage added to the management
> > server, so the primary and secondary storage disks on NFS shares are
> > already available to the management server if it decides to start the
> > instance on another suitable host (of which there are 2).
> >
> > I tried searching some feature like this in Cloudstack administration
> > docs but could not find anything fitting to this scenario.
> >
> > Regards,
> > Parth Patel
> >
>

Re: Cloudstack 4.11 fails to add KVM host

[Parth Patel]

Hi Rohit,

Besides the strange 55 minute gap in the Daznis' agent logs, I have also
encountered a similar situation. As I tried to replicate a failover
scenario, which I touched upon in a previous mail, now the management
server successfully reports agent's state to be "down" and not in alert and
starts an HA-enabled VM in another suitable host. A very strange thing
happens at this time after events of ping timeout: at the time management
server decides to start the VM on another suitable host, the agent host
which has it's LAN cable plugged off, so no network operation can be
performed by it, decides to reboot itself. I think you guys synced the
operations in the code itself, but I don't think a complete reboot of the
host system is required. Also, I have not enabled OOBM for any of my hosts.
But my agent logs do show the line: "Reboot the host" after it has tried to
connect to the management server 4 times. Not necessarily the same issue
that Daznis is facing, but I think its somewhat related and maybe I need
more understanding of the mechanism management server and agents use to
handle a failover scenario.

Regards,
Parth Patel

On Thu, 1 Mar 2018 at 02:38 Rohit Yadav  wrote:

> Hi Daznis,
>
>
> I see the agent decided to kill itself:
>
> 2018-02-28 03:56:34,183 INFO  [cloud.agent.Agent]
> (AgentShutdownThread:null) (logid:) Stopping the agent: Reason = sig.kill
>
>
> This happens when the management server send a ShutDown command to the
> agent. Without logs and additional debugging it will be hard to tell, but
> something went wrong on the management server side. It's possible that the
> mgmt server decided to kill the agent for failure to add the host.
>
>
> You may re-attempt host addition or you can attempt a fresh installation
> and share the details if it fails again.
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> 
> From: Daznis 
> Sent: Wednesday, February 28, 2018 7:13:54 PM
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack 4.11 fails to add KVM host
>
> Hello,
>
>
> Unfortunately I didn't have full debug on. So I can't provide you with
> more detailed logs. I currently wiped both the management and node
> servers and started over. If I will encounter the same issue I will
> provide more detailed logs.
>
>
> Repo:
>
> [cloudstack]
> name=cloudstack
> baseurl=http://cloudstack.apt-get.eu/centos/$releasever/4.11/
> enabled=1
> gpgcheck=0
>
>
> Agent and management log parts attached.
>
>
>
>
>
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> On Wed, Feb 28, 2018 at 1:36 PM, Rohit Yadav 
> wrote:
> > Hi Daznis,
> >
> >
> > Can you share full logs when you're trying to add the KVM host. In the
> logs, do you see something like:
> >
> > SSH command: lsmod|grep kvm
> > Or,
> > Executing cmd: cloudstack-setup-agent  -m 172.20.0.1 -z 1 -p 1 -c 1 -g
> 72f644c9-4f58-3048-8e32-901d38d30672 -a --pubNic=cloudbr0 --prvNic=cloudbr0
> --guestNic=cloudbr0 --hypervisor=kvm
> >
> > I don't see any logs related to c.c.h.k.d.LibvirtServerDiscoverer. Which
> repository did you use to install ACS 4.11?
> >
> >
> > Also, can you share the agent logs (in your KVM host's, from
> /var/log/cloudstack/agent/)?
> >
> >
> > - Rohit
> >
> > <https://cloudstack.apache.org>
> >
> >
> >
> > 
> > From: Daznis 
> > Sent: Wednesday, February 28, 2018 10:42:17 AM
> > To: users@cloudstack.apache.org
> > Subject: Cloudstack 4.11 fails to add KVM host
> >
> > Hello,
> >
> > I'm trying to setup a test VPC cloudstack installation and encountered
> > an error while adding hosts to the cluster. The host sort of appears
> > to be added, but it's not functioning. Trying to add the same host I
> > get an error that host is already present. Both management server and
> > host are running centos 7.4 with latest updates and cloudstack 4.11.
> >
> >
> >
> >
> >
> > management-server.log:
> >
> > 2018-02-28 03:01:47,516 DEBUG [c.c.r.ResourceState]
> > (AgentConnectTaskPool-1:ctx-e30615f2) (logid:541691a9) Resource state
> > update: [id = 1; name = chi-phv01; old state = Creating; event =
> > InternalCreated; new state = Enabled]
> > 2018-02-28 03:01:47,516 DEBUG [c.c.h.Status]
> > (AgentConnectTaskPool-1:ctx-e30615f2) (logid:541691a9)
> > Transition:[Resource state = Enabled, Agent event = AgentConnected,
&g

Re: HA-enabled VM not starting or migrating to another host if current host goes down

[Parth Patel]

Hi Simon,

I'm using KVM hypervisor.

On Sun 25 Feb, 2018, 01:41 Simon Weller,  wrote:

> Which hypervisor are you using?
>
> Simon Weller/615-312-6068
>
> -Original Message-
> From: Parth Patel [parthpatel2...@gmail.com]
> Received: Saturday, 24 Feb 2018, 11:33AM
> To: users@cloudstack.apache.org [users@cloudstack.apache.org]
> Subject: HA-enabled VM not starting or migrating to another host if
> current host goes down
>
> Hi,
>
> I am developing an enterprise-level cloud infrastructure currently using
> Cloudstack 4.9. My head of department wished to check a specific failsafe
> scenario. It is as follows:
>
> A highly available VM whose continuous execution even in event of some
> storage or connection error, should remain running or migrate to another
> host automatically if anything were to happen to the current host. I tried
> removing the LAN cable from the current host after an HA-enabled VM was
> executing on it, but the management server would not auto-start the VM on
> another host. It kept printing error messages such as "Communication
> failure. Host 5 timed out due to even PingTimeout" in management server
> logs. I have manually set the ping timeout duration to 30 seconds and its
> multiplier value to 1.
>
> Any suggestions as to what extra configuration is needed to make Cloudstack
> start the VM on another host or migrate it?
>
> Just to be clear, the host on which the HA-enabled VM is running does not
> have its primary and secondary storage added to the management server, so
> the primary and secondary storage disks on NFS shares are already available
> to the management server if it decides to start the instance on another
> suitable host (of which there are 2).
>
> I tried searching some feature like this in Cloudstack administration docs
> but could not find anything fitting to this scenario.
>
> Regards,
> Parth Patel
>

HA-enabled VM not starting or migrating to another host if current host goes down

[Parth Patel]

Hi,

I am developing an enterprise-level cloud infrastructure currently using
Cloudstack 4.9. My head of department wished to check a specific failsafe
scenario. It is as follows:

A highly available VM whose continuous execution even in event of some
storage or connection error, should remain running or migrate to another
host automatically if anything were to happen to the current host. I tried
removing the LAN cable from the current host after an HA-enabled VM was
executing on it, but the management server would not auto-start the VM on
another host. It kept printing error messages such as "Communication
failure. Host 5 timed out due to even PingTimeout" in management server
logs. I have manually set the ping timeout duration to 30 seconds and its
multiplier value to 1.

Any suggestions as to what extra configuration is needed to make Cloudstack
start the VM on another host or migrate it?

Just to be clear, the host on which the HA-enabled VM is running does not
have its primary and secondary storage added to the management server, so
the primary and secondary storage disks on NFS shares are already available
to the management server if it decides to start the instance on another
suitable host (of which there are 2).

I tried searching some feature like this in Cloudstack administration docs
but could not find anything fitting to this scenario.

Regards,
Parth Patel