Re: [VOTE] Ceph, ZFS or Linux Soft RAID?
Hello, Isn't xfs better than ext4 for that? Thanks. On Mon, Jul 15, 2019 at 06:23:11PM +0700, Ivan Kudryavtsev wrote: > if you use local fs, use just ext4 over the required disk topology which > gives the desired redundancy.
Re: [ANNOUNCE] Apache CloudStack LTS Maintenance Release 4.11.3.0
My congratulations to you, good people! Thank you for doing this great job. I've upgraded yesterday, everything to keep working fine. The only thing that surprised me a little was the fact that one should upgrade the system-VM image to upgrade a minor version. I think, it can be a trap for people who tend to run "yum upgrade" on their servers without diving into the details of each package marked for upgrading. I hope, no one will hurt due to that. :-) Again, thanks to everyone who contributes to this great project. You totally rock! On Sat, Jul 13, 2019 at 04:52:12PM +0100, Paul Angus wrote: > Announcing Apache CloudStack LTS Maintenance Release 4.11.30
Re: Reg : qemu-kvm upgrade in KVM hypervisor agent
Hello, I tried to use qemu-kvm-ev with ACS 4.11.2 and have found out that it makes impossible to take a snapshot of a running instance's volume. :-( On Thu, Jun 13, 2019 at 07:16:24AM +, Pavan Kumar Aravapalli wrote: > Hello, > As part of KVM upgrade it's found that default package 'qemu-kvm' [ which > comes with OS distro] does not support machine chipset type 'q35'. And > this support can be leveraged by installing 'qemu-kvm-ev' which is from > centos-release-qemu-ev repo. > I saw some some old mail threads saying that they have integrated > qemu-kvm-ev, but still in agent bundle rpm requirements > [https://github.com/apache/cloudstack/blob/master/packaging/centos7/cloud.spec] > I found old 'qemu-kvm' only. > Can we leverage this package qemu-kvm-ev in agent rpm bundle?. Has anyone > faced any issues in Cloud Stack after upgrading the package.
Re: K8S LoadBalancer
Thank you very much, Gregor! Would you also be so kind as to help me a little? I tried to deploy your CCM in my K8S, but something seems to be gone wrong. I submitted my question as an issue: https://github.com/swisstxt/cloudstack-cloud-controller-manager/issues/9 - could you, please, take a look? Thanks in advance! On Tue, Jun 11, 2019 at 10:23:20AM +, Riepl, Gregor (SWISS TXT) wrote: > Hi Vladimir > > The CloudStack cloud provider code will be removed from the k8s core > soon: https://github.com/kubernetes/kubernetes/pull/72178 > > It's not advisable to use it any more. > > You can use our CCM that is based on the old cloud provider instead: > https://github.com/swisstxt/cloudstack-cloud-controller-manager > > Regards, > Gregor > > > On Mon, 2019-06-10 at 18:12 +0300, Vladimir Melnik wrote: > > Hello, > > > > Does anyone has experience of using ACS as a LoadBalancer's provider > > for Kubernetes? > > > > As I guess, it's possible (according to these modules' presence: > > https://github.com/kubernetes/kubernetes/tree/master/pkg/cloudprovider/providers/cloudstack > > ), but now I'm looking for some documentation and examples for that. > > > > Thanks in advance for any clues. :-) > > > > Best regards, > > V.Melnyk > > -- V.Melnik
K8S LoadBalancer
Hello, Does anyone has experience of using ACS as a LoadBalancer's provider for Kubernetes? As I guess, it's possible (according to these modules' presence: https://github.com/kubernetes/kubernetes/tree/master/pkg/cloudprovider/providers/cloudstack), but now I'm looking for some documentation and examples for that. Thanks in advance for any clues. :-) Best regards, V.Melnyk
Re: Trying to build the EL6 packages, getting the EL7 ones
Thank you! But I need to perform certain tests on CentOS-6 before to confirm that the patch I proposed is 100% correct. I tested it on CentOS-7, but I also would like to make sure that CentOS-6 will work fine as well. On Mon, Apr 22, 2019 at 02:57:22PM +0100, Richard Lawley wrote: > If you're doing this because the official repo doesn't contain the > latest version then you can use this repo instead to get el6 packages > > http://packages.shapeblue.com/cloudstack/upstream/centos/4.11/ > > On Mon, 22 Apr 2019, 11:04 Vladimir Melnik, wrote: > > > Dear colleagues, > > > > I'm trying to build the CentOS6-compatible packages (using this document: > > http://docs.cloudstack.apache.org/en/latest/installguide/building_from_source.html#building-rpms-from-source), > > running `./package.sh -d centos63`, but the files I get have the "el7" > > suffix in their names: > > cloudstack-agent-4.11.2.0-1.el7.centos.x86_64.rpm > > cloudstack-baremetal-agent-4.11.2.0-1.el7.centos.x86_64.rpm > > cloudstack-cli-4.11.2.0-1.el7.centos.x86_64.rpm > > cloudstack-common-4.11.2.0-1.el7.centos.x86_64.rpm > > cloudstack-integration-tests-4.11.2.0-1.el7.centos.x86_64.rpm > > cloudstack-management-4.11.2.0-1.el7.centos.x86_64.rpm > > cloudstack-marvin-4.11.2.0-1.el7.centos.x86_64.rpm > > cloudstack-usage-4.11.2.0-1.el7.centos.x86_64.rpm > > > > Am I doing it wrong? > > > > Thanks in advance for the clue! > > -- V.Melnik
Trying to build the EL6 packages, getting the EL7 ones
Dear colleagues, I'm trying to build the CentOS6-compatible packages (using this document: http://docs.cloudstack.apache.org/en/latest/installguide/building_from_source.html#building-rpms-from-source), running `./package.sh -d centos63`, but the files I get have the "el7" suffix in their names: cloudstack-agent-4.11.2.0-1.el7.centos.x86_64.rpm cloudstack-baremetal-agent-4.11.2.0-1.el7.centos.x86_64.rpm cloudstack-cli-4.11.2.0-1.el7.centos.x86_64.rpm cloudstack-common-4.11.2.0-1.el7.centos.x86_64.rpm cloudstack-integration-tests-4.11.2.0-1.el7.centos.x86_64.rpm cloudstack-management-4.11.2.0-1.el7.centos.x86_64.rpm cloudstack-marvin-4.11.2.0-1.el7.centos.x86_64.rpm cloudstack-usage-4.11.2.0-1.el7.centos.x86_64.rpm Am I doing it wrong? Thanks in advance for the clue!
Re: [DISCUSS] Remove support for el6 packaging in 4.13/4.14
Hi Rohit, I agree that being stuck to CentOS-6 is not the best strategy and I'd gladly get rid of this old distro in my environments. It could be pretty easy to upgrade the management servers, but upgrading the hosts means "cold" migration from the CentOS6-based cluster to a CentOS7-based one (alas, "hot" migration between different QEMUs won't work, I tried it before). So I'll have to move ~100 VMs between the clusters, which means shutting them all down one by one. That's scary :-) Maybe it would be possible to make ACS "know" what operating system is being run, so ACS could only use the available features? It would be cool. On Sun, Apr 21, 2019 at 08:02:00PM +, Rohit Yadav wrote: > Hi Vladimir, > > > Thanks for sharing your opinion. I can understand the position of our > conservative users such as yourself with respect to a potential upgrade, both > the effort and the risks involved in doing the upgrades. > > > That said, I would like to encourage you and other conservative users to > consider upgrading because it becomes difficult to support older > distributions moving forward especially when CentOS/RHEL 8 will arrive and it > also hinders us to (a) support newer features wrt KVM and (b) forces us to > depend on older distro-provided dependencies that CloudStack uses and > potentially puts a cloud environment under security risks. > > > As Andrija puts it, you can attempt to upgrade to CentOS7 based infra by > doing a fresh installation and move to the new server/hosts in a rolling way. > If you'd need any pointers in that regard I'm sure we can ask Andrija and > others to share some details on how to do that. Meanwhile, you may also refer > to my colleague Dag's talk on upgrade best practices from CCCNA17: > https://www.slideshare.net/ShapeBlue/cccna17-cloudstack-upgrade-best-practicespdf > > > I would like to hear again from you on this and see if we can all agree > towards a decision that benefits all of our community. Thanks. > > > Regards, > > Rohit Yadav > > Software Architect, ShapeBlue > > https://www.shapeblue.com > > > From: Vladimir Melnik > Sent: Friday, April 19, 2019 8:01:20 PM > To: users@cloudstack.apache.org > Cc: d...@cloudstack.apache.org > Subject: Re: [DISCUSS] Remove support for el6 packaging in 4.13/4.14 > > Dear colleagues, > > As for me, I still have 3 hosts running CentOS 6 and 2 management servers in > one of my production environments. > > Also I have 1 management servers running CentOS 6 in another environment. > > If my voice counts, I'd propose -1. > > Thanks for attention :-) > > On Mon, Apr 15, 2019 at 07:44:58AM +, Rohit Yadav wrote: > > All, > > > > > > With CentOS8 around the corner to be released sometime around the summer, I > > would like to propose to deprecate CentOS6 as support management server > > host distro and KVM host distro. Non-systemd enabled Ubuntu releases have > > been already deprecated [1]. > > > > > > The older CentOS6 version would hold us back as we try to adapt, use and > > support newer JRE version, kvm/libvirt version, the Linux kernel, and > > several other older dependencies. Both CentOS6 and RHEL6 have reached EOL > > on May 10th, 2017 wrt full updates [1]. > > > > > > If we don't have any disagreements, I propose we remove el6 packaging > > support in the next major release - 4.13. But, if there are users and > > organisations that will be badly impacted, let 4.13 be the last of releases > > to support el6 and we definitely remove el6 support in 4.14. > > > > What are your thoughts? > > > > > > [1] EOL date wiki reference: > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Hypervisor+and+Management+Server+OS+EOL+Dates > > > > > > > > Regards, > > > > Rohit Yadav > > > > Software Architect, ShapeBlue > > > > https://www.shapeblue.com > > > > rohit.ya...@shapeblue.com > > www.shapeblue.com<http://www.shapeblue.com> > > Amadeus House, Floral Street, London WC2E 9DPUK > > @shapeblue > > > > > > > > -- > V.Melnik > > rohit.ya...@shapeblue.com > www.shapeblue.com > Amadeus House, Floral Street, London WC2E 9DPUK > @shapeblue > > > -- V.Melnik
Re: [DISCUSS] Remove support for el6 packaging in 4.13/4.14
Dear colleagues, As for me, I still have 3 hosts running CentOS 6 and 2 management servers in one of my production environments. Also I have 1 management servers running CentOS 6 in another environment. If my voice counts, I'd propose -1. Thanks for attention :-) On Mon, Apr 15, 2019 at 07:44:58AM +, Rohit Yadav wrote: > All, > > > With CentOS8 around the corner to be released sometime around the summer, I > would like to propose to deprecate CentOS6 as support management server host > distro and KVM host distro. Non-systemd enabled Ubuntu releases have been > already deprecated [1]. > > > The older CentOS6 version would hold us back as we try to adapt, use and > support newer JRE version, kvm/libvirt version, the Linux kernel, and several > other older dependencies. Both CentOS6 and RHEL6 have reached EOL on May > 10th, 2017 wrt full updates [1]. > > > If we don't have any disagreements, I propose we remove el6 packaging support > in the next major release - 4.13. But, if there are users and organisations > that will be badly impacted, let 4.13 be the last of releases to support el6 > and we definitely remove el6 support in 4.14. > > What are your thoughts? > > > [1] EOL date wiki reference: > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Hypervisor+and+Management+Server+OS+EOL+Dates > > > > Regards, > > Rohit Yadav > > Software Architect, ShapeBlue > > https://www.shapeblue.com > > rohit.ya...@shapeblue.com > www.shapeblue.com > Amadeus House, Floral Street, London WC2E 9DPUK > @shapeblue > > > -- V.Melnik
Re: Snapshots on KVM corrupting disk images
Dear colleagues, Yes, that was my PR and my pull-request. Now I would be very grateful for some kind of help from you. Please, be so kind as to describe your cases here: https://github.com/apache/cloudstack/pull/3194 Thank you so much! On Fri, Mar 01, 2019 at 02:00:05PM -0500, Ivan Kudryavtsev wrote: > Hi, Sean, > I saw the PR https://github.com/apache/cloudstack/pull/3194 > which seems covers one of the bugs. Haven't had enough time to dive into > the code to do a review for snapshot-related workflows, but looks like this > PR does the right thing. Hope it will be added to 4.11.3. > > чт, 28 февр. 2019 г. в 17:02, Sean Lair : > > > Hi Ivan, I wanted to respond here and see if you published a PR yet on > > this. > > > > This is a very scary issue for us as customer can snapshot their volumes > > and end up causing corruption - and they blame us. It's already happened - > > luckily we had Storage Array level snapshots in place as a safety net... > > > > Thanks!! > > Sean > > > > -Original Message- > > From: Ivan Kudryavtsev [mailto:kudryavtsev...@bw-sw.com] > > Sent: Sunday, January 27, 2019 7:29 PM > > To: users ; cloudstack-fan < > > cloudstack-...@protonmail.com> > > Cc: dev > > Subject: Re: Snapshots on KVM corrupting disk images > > > > Well, guys. I dived into CS agent scripts, which make volume snapshots and > > found there are no code for suspend/resume and also no code for qemu-agent > > call fsfreeze/fsthaw. I don't see any blockers adding that code yet and try > > to add it in nearest days. If tests go well, I'll publish the PR, which I > > suppose could be integrated into 4.11.3. > > > > пн, 28 янв. 2019 г., 2:45 cloudstack-fan > > cloudstack-...@protonmail.com.invalid: > > > > > Hello Sean, > > > > > > It seems that you've encountered the same issue that I've been facing > > > during the last 5-6 years of using ACS with KVM hosts (see this > > > thread, if you're interested in additional details: > > > https://mail-archives.apache.org/mod_mbox/cloudstack-users/201807.mbox > > > /browser > > > ). > > > > > > I'd like to state that creating snapshots of a running virtual machine > > > is a bit risky. I've implemented some workarounds in my environment, > > > but I'm still not sure that they are 100% effective. > > > > > > I have a couple of questions, if you don't mind. What kind of storage > > > do you use, if it's not a secret? Does you storage use XFS as a > > filesystem? > > > Did you see something like this in your log-files? > > > [***.***] XFS: qemu-kvm(***) possible memory allocation deadlock size > > > 65552 in kmem_realloc (mode:0x250) > > > [***.***] XFS: qemu-kvm(***) possible memory allocation deadlock size > > > 65552 in kmem_realloc (mode:0x250) > > > [***.***] XFS: qemu-kvm(***) possible memory allocation deadlock size > > > 65552 in kmem_realloc (mode:0x250) > > > Did you see any unusual messages in your log-file when the disaster > > > happened? > > > > > > I hope, things will be well. Wish you good luck and all the best! > > > > > > > > > ‐‐‐ Original Message ‐‐‐ > > > On Tuesday, 22 January 2019 18:30, Sean Lair > > wrote: > > > > > > > Hi all, > > > > > > > > We had some instances where VM disks are becoming corrupted when > > > > using > > > KVM snapshots. We are running CloudStack 4.9.3 with KVM on CentOS 7. > > > > > > > > The first time was when someone mass-enabled scheduled snapshots on > > > > a > > > lot of large number VMs and secondary storage filled up. We had to > > > restore all those VM disks... But believed it was just our fault with > > > letting secondary storage fill up. > > > > > > > > Today we had an instance where a snapshot failed and now the disk > > > > image > > > is corrupted and the VM can't boot. here is the output of some commands: > > > > > > > > > > > -- > > > -- > > > -- > > > -- > > > -- > > > -- > > > -- > > > > > > > > > > > [root@cloudkvm02 c3be0ae5-2248-3ed6-a0c7-acffe25cc8d3]# qemu-img > > > > check > > > ./184aa458-9d4b-4c1b-a3c6-23d28ea28e80 > > > > qemu-img: Could not open './184aa458-9d4b-4c1b-a3c6-23d28ea28e80': > > > > Could > > > not read snapshots: File too large > > > > > > > > [root@cloudkvm02 c3be0ae5-2248-3ed6-a0c7-acffe25cc8d3]# qemu-img > > > > info > > > ./184aa458-9d4b-4c1b-a3c6-23d28ea28e80 > > > > qemu-img: Could not open './184aa458-9d4b-4c1b-a3c6-23d28ea28e80': > > > > Could > > > not read snapshots: File too large > > > > > > > > [root@cloudkvm02
Re: [ANNOUNCE] Apache CloudStack LTS Maintenance Release 4.11.2.0
Thank y'all for good news and great work! Is it safe to upgrade from 4.9.x now? This perspective still scares me a little :-) On Mon, Nov 26, 2018 at 03:04:10PM +, Paul Angus wrote: > Announcing Apache CloudStack LTS Maintenance Release 4.11.2.0 -- V.Melnik
vTPM on KVM hosts
Dear colleagues, Are there any plans on implementing TPM pass-through in ACS? As I understand, the KVM hypervisor is able to provide TPM interface to a guest. Thanks! Best regards, Vlad
Re: qemu2 images are being corrupted
Hello,
I have a side question, if you don't mind.
I'm going to implement the checks you were wrote about, but still having some
doubts. What do you think, is it really safe to run "/usr/bin/qemu-img check"
to check an image of a running VM? As I understand, it should be absolutely
safe, as the "check" operation being run without the "-r" parameter doesn't try
to write anything to the file, but I'd prefer to ask this question.
So, is it safe to perform check of the image that is being used by a running VM?
Thanks to all,
Vlad
On Mon, Jul 02, 2018 at 05:21:24AM -0400, cloudstack-fan wrote:
> * but when you run `qemu-img check ...` to check the image, you may see a
> lot of leaked clusters (that's why I'd strongly advice to check each and
> every image one each and every primary storage at least once per hour by a
> script being run by your monitoring system, something kind of `for imagefile
> in $(find /var/lib/libvirt/images -maxdepth 1 -type f); do {
> /usr/bin/qemu-img check "${imagfile}"; if [[ ${?} -ne 0 ]]; then { ... } fi;
> } done`);
Re: [RESULT][VOTE] Apache CloudStack 4.11.1.0
So good news, thank you really much! On Tue, Jun 26, 2018 at 04:09:52PM +, Paul Angus wrote: > After 72 hours, the vote for CloudStack 4.11.1.0 *passes* with > 3 PMC + 2 non-PMC votes. > +1 (PMC / binding) > Rohit Yadav > Paul Angus > Mike Tutkowski > +1 (non binding) > Nicolas Vazquez > Boris Stoyanov > 0 > Rene Moser > -1 > none > Thanks to everyone participating. > I will now prepare the release announcement to go out after 24 hours to give > the mirrors time to catch up. > Kind regards, > Paul Angus -- V.Melnik
Re: Another zone and a new SystemVM template
Thank you, Dag! The SystemVM template has been installed according to the manual (http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.9/management-server/_systemvm.html), the system VMs (CPVM and SSVM) have been deployed, the console works fine. I have another question, if you don't mind. Is there a way to resynchronize the templates between zones? I have many templates in the first zone, and ACS tried to sync them, but there were networking issues, so now the templates' status' are stuck to "No route to host" (https://www.dropbox.com/s/aqahbl3ntre5ixg/acs-templates-zones.png?dl=0). The networking issue has been fixed, but the status isn't being changed and no attempts to resync the templates is being performed. What should I do to reinitialize this process? Thanks a lot! On Tue, Apr 03, 2018 at 08:10:11AM +, Dag Sonstebo wrote: > Yes looks OK Vladimir, the second entry is as expected. Again it looks like > the template has downloaded already, so check your template > zones tab. > > Regards, > Dag Sonstebo > Cloud Architect > ShapeBlue -- V.Melnik
Re: Another zone and a new SystemVM template
I destroyed the new zone and create it again. Haven't enabled it, haven't done anything else, but now I have 2 records in `template_store_ref`: *** 1. row *** id: 41 store_id: 1 template_id: 3 created: 2016-12-08 11:45:54 last_updated: 2018-04-01 19:25:34 job_id: NULL download_pct: 100 size: 322954240 store_role: Image physical_size: 322954240 download_state: DOWNLOADED error_str: NULL local_path: NULL install_path: template/tmpl/1/3/612564dc-8119-4c58-ae4c-df965ee99447.qcow2 url: http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 state: Ready destroyed: 0 is_copy: 0 update_count: 0 ref_cnt: 0 updated: NULL download_url_created: 1970-01-01 00:00:00 download_url: NULL *** 2. row *** id: 4271 store_id: 61 template_id: 3 created: 2018-04-02 16:45:33 last_updated: 2018-04-02 16:45:33 job_id: NULL download_pct: 100 size: 0 store_role: Image physical_size: 0 download_state: DOWNLOADED error_str: NULL local_path: NULL install_path: template/tmpl/1/3/ url: http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 state: Ready destroyed: 0 is_copy: 0 update_count: 0 ref_cnt: 0 updated: NULL download_url_created: NULL download_url: NULL The 2nd record is new, it has different values in the `id`, `store_id`, `created` and `last_updated` fields, so I hope it's OK. :-) On Mon, Apr 02, 2018 at 07:34:08PM +0300, Vladimir Melnik wrote: > Thank you for replying! > > Yes, I've added the new zone, but I didn't make attempts to install the > template. At the same time I see that it shows as "DOWNLOADED" in the > database and in the list of zones where template is installed. At the same > time I don't see any files in my secondary storage's directory (it's being > mounted and umounted either from the management servers and virtualization > hosts without any errors). > > Perhaps I should destroy and create the zone to make sure this case won't > resolve to any glitches... > > On Mon, Apr 02, 2018 at 04:26:58PM +, Dag Sonstebo wrote: > > Hi Vladimir, > > > > It won’t cause any damage – but if you are concerned just do a mysqldump > > beforehand. > > > > One thing though – have you already added the new zone and secondary > > storage pool? If store ID 41 is your new secondary pool then the second > > entry in your template_store_ref shows the template already downloaded? > > > > Regards, > > Dag Sonstebo > > Cloud Architect > > ShapeBlue > > > > On 02/04/2018, 17:05, "Vladimir Melnik" <v.mel...@uplink.ua> wrote: > > > > Thank you, Dag! > > > > Wouldn't it damage the existing records? > > > > I have the following one in the `vm_template` table: > > > > *** 1. row *** > > id: 3 > > unique_name: routing-3 > > name: SystemVM Template (KVM) > > uuid: b51f9ef8-bd2f-11e6-b74f-06973a00088a > > public: 0 > > featured: 0 > > type: SYSTEM > > hvm: 0 > > bits: 64 > > url: > > http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 > > format: QCOW2 > > created: 2016-12-08 12:18:48 > > removed: NULL > > account_id: 1 > > checksum: c059b0d051e0cd6fbe9d5d4fc40c7e5d > > display_text: SystemVM Template (KVM) > > enable_password: 0 > >enable_sshkey: 0 > > guest_os_id: 15 > > bootable: 1 > > prepopulate: 0 > > cross_zones: 1 > > extractable: 0 > > hypervisor_type: KVM > > source_template_id: NULL > > template_tag: NULL > > sort_key: 0 > > size: 322954240 > >state: Active > > update_count: 0 > > updat
Re: Another zone and a new SystemVM template
Thank you for replying! Yes, I've added the new zone, but I didn't make attempts to install the template. At the same time I see that it shows as "DOWNLOADED" in the database and in the list of zones where template is installed. At the same time I don't see any files in my secondary storage's directory (it's being mounted and umounted either from the management servers and virtualization hosts without any errors). Perhaps I should destroy and create the zone to make sure this case won't resolve to any glitches... On Mon, Apr 02, 2018 at 04:26:58PM +, Dag Sonstebo wrote: > Hi Vladimir, > > It won’t cause any damage – but if you are concerned just do a mysqldump > beforehand. > > One thing though – have you already added the new zone and secondary storage > pool? If store ID 41 is your new secondary pool then the second entry in your > template_store_ref shows the template already downloaded? > > Regards, > Dag Sonstebo > Cloud Architect > ShapeBlue > > On 02/04/2018, 17:05, "Vladimir Melnik" <v.mel...@uplink.ua> wrote: > > Thank you, Dag! > > Wouldn't it damage the existing records? > > I have the following one in the `vm_template` table: > > *** 1. row *** > id: 3 > unique_name: routing-3 > name: SystemVM Template (KVM) > uuid: b51f9ef8-bd2f-11e6-b74f-06973a00088a > public: 0 > featured: 0 > type: SYSTEM > hvm: 0 > bits: 64 > url: > http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 > format: QCOW2 > created: 2016-12-08 12:18:48 > removed: NULL > account_id: 1 > checksum: c059b0d051e0cd6fbe9d5d4fc40c7e5d > display_text: SystemVM Template (KVM) > enable_password: 0 >enable_sshkey: 0 > guest_os_id: 15 > bootable: 1 > prepopulate: 0 > cross_zones: 1 > extractable: 0 > hypervisor_type: KVM > source_template_id: NULL > template_tag: NULL > sort_key: 0 > size: 322954240 >state: Active > update_count: 0 > updated: NULL > dynamically_scalable: 0 > > And also there are 2 ones in the `template_store_ref` table: > > *** 1. row *** > id: 41 > store_id: 1 > template_id: 3 > created: 2016-12-08 11:45:54 > last_updated: 2018-04-01 19:25:34 > job_id: NULL > download_pct: 100 > size: 322954240 > store_role: Image >physical_size: 322954240 > download_state: DOWNLOADED >error_str: NULL > local_path: NULL > install_path: > template/tmpl/1/3/612564dc-8119-4c58-ae4c-df965ee99447.qcow2 > url: > http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 >state: Ready >destroyed: 0 > is_copy: 0 > update_count: 0 > ref_cnt: 0 > updated: NULL > download_url_created: 1970-01-01 00:00:00 > download_url: NULL > *** 2. row *** > id: 4151 > store_id: 41 > template_id: 3 > created: 2018-04-02 14:43:49 > last_updated: 2018-04-02 14:43:49 > job_id: NULL > download_pct: 100 > size: 0 > store_role: Image >physical_size: 0 > download_state: DOWNLOADED >error_str: NULL > local_path: NULL > install_path: template/tmpl/1/3/ > url: > http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 >state: Ready >destroyed: 0 > is_copy: 0 > update_count: 0 > ref_cnt: 0 > updated: NULL > download_url_created: NULL > download_url: NULL > > Maybe it's stupid, but I'm afraid that the script will make changes to > the record #41 instead of upda
Re: Another zone and a new SystemVM template
Thank you, Dag! Wouldn't it damage the existing records? I have the following one in the `vm_template` table: *** 1. row *** id: 3 unique_name: routing-3 name: SystemVM Template (KVM) uuid: b51f9ef8-bd2f-11e6-b74f-06973a00088a public: 0 featured: 0 type: SYSTEM hvm: 0 bits: 64 url: http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 format: QCOW2 created: 2016-12-08 12:18:48 removed: NULL account_id: 1 checksum: c059b0d051e0cd6fbe9d5d4fc40c7e5d display_text: SystemVM Template (KVM) enable_password: 0 enable_sshkey: 0 guest_os_id: 15 bootable: 1 prepopulate: 0 cross_zones: 1 extractable: 0 hypervisor_type: KVM source_template_id: NULL template_tag: NULL sort_key: 0 size: 322954240 state: Active update_count: 0 updated: NULL dynamically_scalable: 0 And also there are 2 ones in the `template_store_ref` table: *** 1. row *** id: 41 store_id: 1 template_id: 3 created: 2016-12-08 11:45:54 last_updated: 2018-04-01 19:25:34 job_id: NULL download_pct: 100 size: 322954240 store_role: Image physical_size: 322954240 download_state: DOWNLOADED error_str: NULL local_path: NULL install_path: template/tmpl/1/3/612564dc-8119-4c58-ae4c-df965ee99447.qcow2 url: http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 state: Ready destroyed: 0 is_copy: 0 update_count: 0 ref_cnt: 0 updated: NULL download_url_created: 1970-01-01 00:00:00 download_url: NULL *** 2. row *** id: 4151 store_id: 41 template_id: 3 created: 2018-04-02 14:43:49 last_updated: 2018-04-02 14:43:49 job_id: NULL download_pct: 100 size: 0 store_role: Image physical_size: 0 download_state: DOWNLOADED error_str: NULL local_path: NULL install_path: template/tmpl/1/3/ url: http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 state: Ready destroyed: 0 is_copy: 0 update_count: 0 ref_cnt: 0 updated: NULL download_url_created: NULL download_url: NULL Maybe it's stupid, but I'm afraid that the script will make changes to the record #41 instead of updating the record #4151. How will it know that we need to install the template to the store #41 (a new secondary storage of a new zone)? Best regards, Vlad On Mon, Apr 02, 2018 at 03:42:38PM +, Dag Sonstebo wrote: > Vladimir, > > Secondary storage is always zone wide, and if you have multiple zones then > you also select which zone to add the new secondary storage pool to. > So as long as you prepopulate the template as per documentation you will be > fine. > > Regards, > Dag Sonstebo > Cloud Architect > ShapeBlue > > On 02/04/2018, 16:37, "Vladimir Melnik" <v.mel...@uplink.ua> wrote: > > Hello, > > It might be a bit stupid question, but I'd like to ask for some help :) > > How to add a new SystemVM template to a new zone? The documentation > (http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.9/management-server/_systemvm.html) > is quite clear, but I can't see how will ACS understand that the template is > being added to a new zone. Should I indicate the zone's ID somehow? How to do > that? > > Thanks in advance! > > -- > V.Melnik > > > > dag.sonst...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > -- V.Melnik
Another zone and a new SystemVM template
Hello, It might be a bit stupid question, but I'd like to ask for some help :) How to add a new SystemVM template to a new zone? The documentation (http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.9/management-server/_systemvm.html) is quite clear, but I can't see how will ACS understand that the template is being added to a new zone. Should I indicate the zone's ID somehow? How to do that? Thanks in advance! -- V.Melnik
Re: Time-out when creating a template from a snapshot
Thanks for sharing!
I think, it's not only an SQL-related issue. I raised timeout thresholds of
haproxy, so now there's no DB exceptions since that, but something is
terminating the "cp" process on the SSVM leaving the template incomplete. There
are no messages about the database in the log-file and ACS thinks that the
operation has been finished successfully.
I'm pretty sure that haproxy was a half of a problem, but the second half is
somewhere inside of the SSVM.
On Thu, Feb 01, 2018 at 09:17:51PM +0100, Andrija Panic wrote:
> Vladimir,
>
> the original error seems as MySQL timeout for sure (I assume because of
> HAPROXY in the middle), and we also had this setup originally (MGMT server
> using HAPROXY on top of galera nodes...) but this has confirmed to be
> issue, no matter what we changed on HAproxy or Mysql, and at that time we
> didn't find the solution (MGMT then was set to hit first galera node
> directly...poor man solution) - problem is that it seems when snapshot
> starts (and can take even hours / or conversion to template, same thing...)
> Java keeps the DB connection/transaction open for all time (which is
> strange approach in my head, for such long image-converting actions)
>
> If I'm not wrong, the snapshot to template conversion should be done via
> agent node, not SSVM ?
> Ping here if you find solution.
>
> Btw, for some actions with images the real timeout = 2 x wait parameter :)
> so change that to 2000 and check if actions fails after 4000 sec.
>
>
>
> On 1 February 2018 at 13:00, Vladimir Melnik <v.mel...@uplink.ua> wrote:
>
> > Thanks a lot, any help will be so much appreciated!
> >
> > On Wed, Jan 31, 2018 at 05:23:25PM +, Nux! wrote:
> > > It's possible there are timeouts being hit somewhere. I'd take this to
> > dev@ to be honest, I am not very familiar with the ssvm internals.
> > >
> > > --
> > > Sent from the Delta quadrant using Borg technology!
> > >
> > > Nux!
> > > www.nux.ro
> > >
> > > - Original Message -
> > > > From: "Vladimir Melnik" <v.mel...@uplink.ua>
> > > > To: "users" <users@cloudstack.apache.org>
> > > > Sent: Wednesday, 31 January, 2018 12:42:01
> > > > Subject: Re: Time-out when creating a template from a snapshot
> > >
> > > > No, it doesn't seem to be a database-related issue.
> > > >
> > > > This time I haven't got any error messages at all. Moreover, I see
> > this template
> > > > as available in the templates' list and there's the following message
> > in the
> > > > log-file:
> > > > 2018-01-31 14:30:09,862 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > > (API-Job-Executor-1:ctx-b39838a2 job-1485421 ctx-3b9d9083)
> > (logid:cb74bce4)
> > > > Complete async job-1485421, jobStatus: SUCCEEDED, resultCode: 0,
> > result:
> > > > org.apache.cloudstack.api.response.TemplateResponse/
> > template/{"id":"b69c65b3-70d6-4000-934c-fac9e887d3ef","name"
> > :"Tucha#2018012713000408","displaytext":"Tucha#
> > 2018012713000408","ispublic":false,"created":"2018-01-
> > 31T13:30:09+0200","isready":true,"passwordenabled":true,"
> > format":"QCOW2","isfeatured":false,"crossZones":false,"
> > ostypeid":"b5490e1c-bd31-11e6-b74f-06973a00088a","ostypename":"Windows
> > > > Server 2012 R2
> > > > (64-bit)","account":"admin#lite","zoneid":"c8d773fa-76ca-
> > 4637-8ecf-88656444fc86","zonename":"z2.tucha13.net","status":"Download
> > > > Complete","size":375809638400,"templatetype":"USER","
> > hypervisor":"KVM","domain":"ROOT","domainid":"b514ef44-
> > bd2f-11e6-b74f-06973a00088a","isextractable":false,"
> > sourcetemplateid":"ba26d2a9-5e2f-468d-8a38-df71a7811ee8","details":{"
> > memoryOvercommitRatio":"1.0","cpuNumber":"4","cpuSpeed":"2399","Message.
> > ReservedCapacityFreed.Flag":"false","cpuOvercommitRatio":"
> > 10","memory":"12288"},"sshkeyenabled":false,"
> > isdynamicallyscalable":false,"tags":[]}
> > > >
> > > > But at the same time I see that th
Re: Time-out when creating a template from a snapshot
Thanks a lot, any help will be so much appreciated!
On Wed, Jan 31, 2018 at 05:23:25PM +, Nux! wrote:
> It's possible there are timeouts being hit somewhere. I'd take this to dev@
> to be honest, I am not very familiar with the ssvm internals.
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -----
> > From: "Vladimir Melnik" <v.mel...@uplink.ua>
> > To: "users" <users@cloudstack.apache.org>
> > Sent: Wednesday, 31 January, 2018 12:42:01
> > Subject: Re: Time-out when creating a template from a snapshot
>
> > No, it doesn't seem to be a database-related issue.
> >
> > This time I haven't got any error messages at all. Moreover, I see this
> > template
> > as available in the templates' list and there's the following message in the
> > log-file:
> > 2018-01-31 14:30:09,862 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-b39838a2 job-1485421 ctx-3b9d9083) (logid:cb74bce4)
> > Complete async job-1485421, jobStatus: SUCCEEDED, resultCode: 0, result:
> > org.apache.cloudstack.api.response.TemplateResponse/template/{"id":"b69c65b3-70d6-4000-934c-fac9e887d3ef","name":"Tucha#2018012713000408","displaytext":"Tucha#2018012713000408","ispublic":false,"created":"2018-01-31T13:30:09+0200","isready":true,"passwordenabled":true,"format":"QCOW2","isfeatured":false,"crossZones":false,"ostypeid":"b5490e1c-bd31-11e6-b74f-06973a00088a","ostypename":"Windows
> > Server 2012 R2
> > (64-bit)","account":"admin#lite","zoneid":"c8d773fa-76ca-4637-8ecf-88656444fc86","zonename":"z2.tucha13.net","status":"Download
> > Complete","size":375809638400,"templatetype":"USER","hypervisor":"KVM","domain":"ROOT","domainid":"b514ef44-bd2f-11e6-b74f-06973a00088a","isextractable":false,"sourcetemplateid":"ba26d2a9-5e2f-468d-8a38-df71a7811ee8","details":{"memoryOvercommitRatio":"1.0","cpuNumber":"4","cpuSpeed":"2399","Message.ReservedCapacityFreed.Flag":"false","cpuOvercommitRatio":"10","memory":"12288"},"sshkeyenabled":false,"isdynamicallyscalable":false,"tags":[]}
> >
> > But at the same time I see that the template's file is less than the
> > snapshot's
> > file:
> > -rw-r--r-- 1 root root 311253204992 Jan 31 00:14
> > /mnt/SecStorage/ea7ebf9a-5195-31ab-be8e-f9348f9fee2b/snapshots/4391/12401/e7364ecf-56f2-451d-ba2e-537b9465097f
> > -rw-r--r-- 1 root root 195583541248 Jan 31 12:30
> > /mnt/SecStorage/ea7ebf9a-5195-31ab-be8e-f9348f9fee2b/template/tmpl/3121/473/e7364ecf-56f2-451d-ba2e-537b9465097f.qcow2
> >
> > The oddest thing is that the "cp" process in the SSVM is being terminated
> > exactly in an hour after its start. Who would be doing that each time I'm
> > trying to create a template? Isn't it being done by some script at the SSVM
> > itself?
> >
> >
> > On Mon, Jan 29, 2018 at 05:36:54PM +0200, Vladimir Melnik wrote:
> >> Thank you, Lucian! My MySQL timeout thresolds are higher than 1 hour, but
> >> there's HAproxy between ACS and MySQL, so I've changed haproxy's timeouts
> >> and
> >> now will see what happens in an hour :-)
> >>
> >> On Mon, Jan 29, 2018 at 11:47:31AM +, Nux! wrote:
> >> > I'm usually a sucker with these Java errors, but the error coming from
> >> > the jdbc
> >> > mysql driver makes me think maybe this is related to MySQL timeouts.
> >> >
> >> > Can you check your db installation for wait_timeouts,
> >> > interactive_timeout,
> >> > connect_timeout and so on, see if any match your 3600 seconds?
> >> >
> >> > random search result
> >> > http://www.supermanhamuerto.com/doku.php?id=java%3athelastpacketsuccessfullyreceivedfromserver
> >> >
> >> > hth
> >> > Lucian
> >> >
> >> > --
> >> > Sent from the Delta quadrant using Borg technology!
> >> >
> >> > Nux!
> >> > www.nux.ro
> >> >
> >> > - Original Message -
> >> > > From: "Vladimir Melnik" <v.mel...@uplink.ua>
Re: Time-out when creating a template from a snapshot
No, the primary storage is local (in this case), but the primary storage isn't
being involved, as I'm creating a template from a snapshot which resides on a
secondary storage.
The snapshot's size is ~300GB.
On Wed, Jan 31, 2018 at 06:18:43PM +, Simon Weller wrote:
> Is your primary storage NFS as well? How big is the disk being snapshotted?
>
>
>
> From: Nux! <n...@li.nux.ro>
> Sent: Wednesday, January 31, 2018 11:23 AM
> To: users
> Subject: Re: Time-out when creating a template from a snapshot
>
> It's possible there are timeouts being hit somewhere. I'd take this to dev@
> to be honest, I am not very familiar with the ssvm internals.
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Vladimir Melnik" <v.mel...@uplink.ua>
> > To: "users" <users@cloudstack.apache.org>
> > Sent: Wednesday, 31 January, 2018 12:42:01
> > Subject: Re: Time-out when creating a template from a snapshot
>
> > No, it doesn't seem to be a database-related issue.
> >
> > This time I haven't got any error messages at all. Moreover, I see this
> > template
> > as available in the templates' list and there's the following message in the
> > log-file:
> > 2018-01-31 14:30:09,862 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-1:ctx-b39838a2 job-1485421 ctx-3b9d9083) (logid:cb74bce4)
> > Complete async job-1485421, jobStatus: SUCCEEDED, resultCode: 0, result:
> > org.apache.cloudstack.api.response.TemplateResponse/template/{"id":"b69c65b3-70d6-4000-934c-fac9e887d3ef","name":"Tucha#2018012713000408","displaytext":"Tucha#2018012713000408","ispublic":false,"created":"2018-01-31T13:30:09+0200","isready":true,"passwordenabled":true,"format":"QCOW2","isfeatured":false,"crossZones":false,"ostypeid":"b5490e1c-bd31-11e6-b74f-06973a00088a","ostypename":"Windows
> > Server 2012 R2
> > (64-bit)","account":"admin#lite","zoneid":"c8d773fa-76ca-4637-8ecf-88656444fc86","zonename":"z2.tucha13.net","status":"Download
> > Complete","size":375809638400,"templatetype":"USER","hypervisor":"KVM","domain":"ROOT","domainid":"b514ef44-bd2f-11e6-b74f-06973a00088a","isextractable":false,"sourcetemplateid":"ba26d2a9-5e2f-468d-8a38-df71a7811ee8","details":{"memoryOvercommitRatio":"1.0","cpuNumber":"4","cpuSpeed":"2399","Message.ReservedCapacityFreed.Flag":"false","cpuOvercommitRatio":"10","memory":"12288"},"sshkeyenabled":false,"isdynamicallyscalable":false,"tags":[]}
> >
> > But at the same time I see that the template's file is less than the
> > snapshot's
> > file:
> > -rw-r--r-- 1 root root 311253204992 Jan 31 00:14
> > /mnt/SecStorage/ea7ebf9a-5195-31ab-be8e-f9348f9fee2b/snapshots/4391/12401/e7364ecf-56f2-451d-ba2e-537b9465097f
> > -rw-r--r-- 1 root root 195583541248 Jan 31 12:30
> > /mnt/SecStorage/ea7ebf9a-5195-31ab-be8e-f9348f9fee2b/template/tmpl/3121/473/e7364ecf-56f2-451d-ba2e-537b9465097f.qcow2
> >
> > The oddest thing is that the "cp" process in the SSVM is being terminated
> > exactly in an hour after its start. Who would be doing that each time I'm
> > trying to create a template? Isn't it being done by some script at the SSVM
> > itself?
> >
> >
> > On Mon, Jan 29, 2018 at 05:36:54PM +0200, Vladimir Melnik wrote:
> >> Thank you, Lucian! My MySQL timeout thresolds are higher than 1 hour, but
> >> there's HAproxy between ACS and MySQL, so I've changed haproxy's timeouts
> >> and
> >> now will see what happens in an hour :-)
> >>
> >> On Mon, Jan 29, 2018 at 11:47:31AM +, Nux! wrote:
> >> > I'm usually a sucker with these Java errors, but the error coming from
> >> > the jdbc
> >> > mysql driver makes me think maybe this is related to MySQL timeouts.
> >> >
> >> > Can you check your db installation for wait_timeouts,
> >> > interactive_timeout,
> >> > connect_timeout and so on, see if any match your 3600 seconds?
> >> >
> >> > random search result
> >> > http://www.supermanhamuerto.com/doku.
Re: Time-out when creating a template from a snapshot
No, it doesn't seem to be a database-related issue.
This time I haven't got any error messages at all. Moreover, I see this
template as available in the templates' list and there's the following message
in the log-file:
2018-01-31 14:30:09,862 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-1:ctx-b39838a2 job-1485421 ctx-3b9d9083) (logid:cb74bce4)
Complete async job-1485421, jobStatus: SUCCEEDED, resultCode: 0, result:
org.apache.cloudstack.api.response.TemplateResponse/template/{"id":"b69c65b3-70d6-4000-934c-fac9e887d3ef","name":"Tucha#2018012713000408","displaytext":"Tucha#2018012713000408","ispublic":false,"created":"2018-01-31T13:30:09+0200","isready":true,"passwordenabled":true,"format":"QCOW2","isfeatured":false,"crossZones":false,"ostypeid":"b5490e1c-bd31-11e6-b74f-06973a00088a","ostypename":"Windows
Server 2012 R2
(64-bit)","account":"admin#lite","zoneid":"c8d773fa-76ca-4637-8ecf-88656444fc86","zonename":"z2.tucha13.net","status":"Download
Complete","size":375809638400,"templatetype":"USER","hypervisor":"KVM","domain":"ROOT","domainid":"b514ef44-bd2f-11e6-b74f-06973a00088a","isextractable":false,"sourcetemplateid":"ba26d2a9-5e2f-468d-8a38-df71a7811ee8","details":{"memoryOvercommitRatio":"1.0","cpuNumber":"4","cpuSpeed":"2399","Message.ReservedCapacityFreed.Flag":"false","cpuOvercommitRatio":"10","memory":"12288"},"sshkeyenabled":false,"isdynamicallyscalable":false,"tags":[]}
But at the same time I see that the template's file is less than the snapshot's
file:
-rw-r--r-- 1 root root 311253204992 Jan 31 00:14
/mnt/SecStorage/ea7ebf9a-5195-31ab-be8e-f9348f9fee2b/snapshots/4391/12401/e7364ecf-56f2-451d-ba2e-537b9465097f
-rw-r--r-- 1 root root 195583541248 Jan 31 12:30
/mnt/SecStorage/ea7ebf9a-5195-31ab-be8e-f9348f9fee2b/template/tmpl/3121/473/e7364ecf-56f2-451d-ba2e-537b9465097f.qcow2
The oddest thing is that the "cp" process in the SSVM is being terminated
exactly in an hour after its start. Who would be doing that each time I'm
trying to create a template? Isn't it being done by some script at the SSVM
itself?
On Mon, Jan 29, 2018 at 05:36:54PM +0200, Vladimir Melnik wrote:
> Thank you, Lucian! My MySQL timeout thresolds are higher than 1 hour, but
> there's HAproxy between ACS and MySQL, so I've changed haproxy's timeouts and
> now will see what happens in an hour :-)
>
> On Mon, Jan 29, 2018 at 11:47:31AM +, Nux! wrote:
> > I'm usually a sucker with these Java errors, but the error coming from the
> > jdbc mysql driver makes me think maybe this is related to MySQL timeouts.
> >
> > Can you check your db installation for wait_timeouts, interactive_timeout,
> > connect_timeout and so on, see if any match your 3600 seconds?
> >
> > random search result
> > http://www.supermanhamuerto.com/doku.php?id=java%3athelastpacketsuccessfullyreceivedfromserver
> >
> > hth
> > Lucian
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > - Original Message -
> > > From: "Vladimir Melnik" <v.mel...@uplink.ua>
> > > To: "users" <users@cloudstack.apache.org>
> > > Sent: Monday, 29 January, 2018 09:29:18
> > > Subject: Time-out when creating a template from a snapshot
> >
> > > Dear colleagues,
> > >
> > > Would anyone be so kind as to help me to find out how to change time
> > > limits for
> > > template creation?
> > >
> > > When I create a template from a snapshot, I have only an hour to have it
> > > done,
> > > othewise the operation is being terminated exaxtly after 3600 seconds,
> > > but I
> > > can't understand why does it happen, as my settings seem to be quite
> > > "loose":
> > >
> > > create.private.template.from.snapshot.wait = 10800
> > > secstorage.cmd.execution.time.max = 240
> > >vm.job.timeout = 60
> > > wait = 1800
> > >
> > > Here are the messages I see in the management log-file:
> > >
> > > 2018-01-29 10:22:04,029 WARN [o.a.c.f.j.i.AsyncJobMonitor]
&
Re: Time-out when creating a template from a snapshot
Thank you, Lucian! My MySQL timeout thresolds are higher than 1 hour, but
there's HAproxy between ACS and MySQL, so I've changed haproxy's timeouts and
now will see what happens in an hour :-)
On Mon, Jan 29, 2018 at 11:47:31AM +, Nux! wrote:
> I'm usually a sucker with these Java errors, but the error coming from the
> jdbc mysql driver makes me think maybe this is related to MySQL timeouts.
>
> Can you check your db installation for wait_timeouts, interactive_timeout,
> connect_timeout and so on, see if any match your 3600 seconds?
>
> random search result
> http://www.supermanhamuerto.com/doku.php?id=java%3athelastpacketsuccessfullyreceivedfromserver
>
> hth
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Vladimir Melnik" <v.mel...@uplink.ua>
> > To: "users" <users@cloudstack.apache.org>
> > Sent: Monday, 29 January, 2018 09:29:18
> > Subject: Time-out when creating a template from a snapshot
>
> > Dear colleagues,
> >
> > Would anyone be so kind as to help me to find out how to change time limits
> > for
> > template creation?
> >
> > When I create a template from a snapshot, I have only an hour to have it
> > done,
> > othewise the operation is being terminated exaxtly after 3600 seconds, but I
> > can't understand why does it happen, as my settings seem to be quite
> > "loose":
> >
> > create.private.template.from.snapshot.wait = 10800
> > secstorage.cmd.execution.time.max = 240
> >vm.job.timeout = 60
> > wait = 1800
> >
> > Here are the messages I see in the management log-file:
> >
> > 2018-01-29 10:22:04,029 WARN [o.a.c.f.j.i.AsyncJobMonitor]
> > (Timer-1:ctx-7a53941f) (logid:e215433a) Task (job-1476131) has been pending
> > for
> > 3577 seconds
> > 2018-01-29 10:22:26,836 DEBUG [c.c.a.t.Request]
> > (API-Job-Executor-1:ctx-1727ccb3
> > job-1476131 ctx-51926357) (logid:a5fbfcf6) Seq 21-5880575213439025165:
> > Received: { Ans: , MgmtId: 7477638727846, via: 21(s-1-VM), Ver: v1, Flags:
> > 110, { CopyCmdAnswer } }
> > 2018-01-29 10:22:26,838 DEBUG [o.a.c.s.i.s.TemplateObject]
> > (API-Job-Executor-1:ctx-1727ccb3 job-1476131 ctx-51926357) (logid:a5fbfcf6)
> > failed to process event and answer
> > com.cloud.utils.exception.CloudRuntimeException: DB Exception on:
> > com.mysql.jdbc.JDBC4PreparedStatement@1e42ea31: SELECT
> > template_store_ref.id,
> > template_store_ref.store_id, template_store_ref.template_id,
> > template_store_ref.store_role, template_store_ref.created,
> > template_store_ref.last_updated, template_store_ref.download_pct,
> > template_store_ref.size, template_store_ref.physical_size,
> > template_store_ref.download_state, template_store_ref.local_path,
> > template_store_ref.error_str, template_store_ref.job_id,
> > template_store_ref.install_path, template_store_ref.url,
> > template_store_ref.download_url, template_store_ref.download_url_created,
> > template_store_ref.is_copy, template_store_ref.destroyed,
> > template_store_ref.update_count, template_store_ref.updated,
> > template_store_ref.state, template_store_ref.ref_cnt FROM template_store_ref
> > WHERE template_store_ref.store_id = 1 AND template_store_ref.template_id =
> > 471
> > AND template_store_ref.destroyed = 0 ORDER BY RAND() LIMIT 1
> >at
> >
> > com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:427)
> >at
> >
> > com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:363)
> >at
> >
> > com.cloud.utils.db.GenericDaoBase.findOneIncludingRemovedBy(GenericDaoBase.java:891)
> >at
> >
> > org.apache.cloudstack.storage.image.db.TemplateDataStoreDaoImpl.findByStoreTemplate(TemplateDataStoreDaoImpl.java:356)
> >at sun.reflect.GeneratedMethodAccessor81.invoke(Unknown Source)
> >at
> >
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >at java.lang.reflect.Method.invoke(Method.java:606)
> >at
> >
> > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> >at
> >
> > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodIn
Time-out when creating a template from a snapshot
Dear colleagues,
Would anyone be so kind as to help me to find out how to change time limits for
template creation?
When I create a template from a snapshot, I have only an hour to have it done,
othewise the operation is being terminated exaxtly after 3600 seconds, but I
can't understand why does it happen, as my settings seem to be quite "loose":
create.private.template.from.snapshot.wait = 10800
secstorage.cmd.execution.time.max = 240
vm.job.timeout = 60
wait = 1800
Here are the messages I see in the management log-file:
2018-01-29 10:22:04,029 WARN [o.a.c.f.j.i.AsyncJobMonitor]
(Timer-1:ctx-7a53941f) (logid:e215433a) Task (job-1476131) has been pending for
3577 seconds
2018-01-29 10:22:26,836 DEBUG [c.c.a.t.Request]
(API-Job-Executor-1:ctx-1727ccb3 job-1476131 ctx-51926357) (logid:a5fbfcf6) Seq
21-5880575213439025165: Received: { Ans: , MgmtId: 7477638727846, via:
21(s-1-VM), Ver: v1, Flags: 110, { CopyCmdAnswer } }
2018-01-29 10:22:26,838 DEBUG [o.a.c.s.i.s.TemplateObject]
(API-Job-Executor-1:ctx-1727ccb3 job-1476131 ctx-51926357) (logid:a5fbfcf6)
failed to process event and answer
com.cloud.utils.exception.CloudRuntimeException: DB Exception on:
com.mysql.jdbc.JDBC4PreparedStatement@1e42ea31: SELECT template_store_ref.id,
template_store_ref.store_id, template_store_ref.template_id,
template_store_ref.store_role, template_store_ref.created,
template_store_ref.last_updated, template_store_ref.download_pct,
template_store_ref.size, template_store_ref.physical_size,
template_store_ref.download_state, template_store_ref.local_path,
template_store_ref.error_str, template_store_ref.job_id,
template_store_ref.install_path, template_store_ref.url,
template_store_ref.download_url, template_store_ref.download_url_created,
template_store_ref.is_copy, template_store_ref.destroyed,
template_store_ref.update_count, template_store_ref.updated,
template_store_ref.state, template_store_ref.ref_cnt FROM template_store_ref
WHERE template_store_ref.store_id = 1 AND template_store_ref.template_id = 471
AND template_store_ref.destroyed = 0 ORDER BY RAND() LIMIT 1
at
com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:427)
at
com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:363)
at
com.cloud.utils.db.GenericDaoBase.findOneIncludingRemovedBy(GenericDaoBase.java:891)
at
org.apache.cloudstack.storage.image.db.TemplateDataStoreDaoImpl.findByStoreTemplate(TemplateDataStoreDaoImpl.java:356)
at sun.reflect.GeneratedMethodAccessor81.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy102.findByStoreTemplate(Unknown Source)
at
org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:209)
at
org.apache.cloudstack.storage.image.TemplateServiceImpl.copyTemplateCallBack(TemplateServiceImpl.java:951)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.cloudstack.framework.async.AsyncCallbackDispatcher.dispatch(AsyncCallbackDispatcher.java:148)
at
org.apache.cloudstack.framework.async.InplaceAsyncCallbackDriver.performCompletionCallback(InplaceAsyncCallbackDriver.java:25)
at
org.apache.cloudstack.framework.async.AsyncCallbackDispatcher.complete(AsyncCallbackDispatcher.java:126)
at
org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:457)
at
org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:68)
at
CVE-2017-5754: perofmance levels after patching the kernel
Dear colleagues, Have you already upgraded? How do KVM-driven virtual machines feel? Did you measure or observe any changes? What does your Zabbix/Munin/etc. show? Thanks for sharing your experience. -- Best regards, V.Melnyk
Re: Where is the vm root password published?
On the other side, it would mean that you've got to have this key installed on
each of your API-client (even to the web-client), as without this key the
frontend app couldn't decrypt the password that is being sent when a client
deploys a new VM or changes the VM's password. :-)
On Tue, Nov 28, 2017 at 02:18:03PM +0200, Vladimir Melnik wrote:
> Aye, should be cool to have them encrypted by some RSA-key that would be
> installed to the VM's template.
>
> Though at this moment one should keep an eye on the systems where these logs
> are stored.
>
> On Tue, Nov 28, 2017 at 03:39:55PM +0530, Makrand wrote:
> > Assuming all the passwords appearing in logs must be masked (kind of
> > encrypted) How does one decrypt those password from logs?
> >
> > BTW, if passwords are just logged as plain text (even for temp amount of
> > time), or stored as plain text over VR, then that's not a very secure
> > thing, is it??
> >
> > --
> > Makrand
> >
> >
> > On Tue, Nov 28, 2017 at 2:58 PM, Vladimir Melnik <v.mel...@uplink.ua> wrote:
> >
> > > Hello,
> > >
> > > Would you mind if I share a sample line from the log-file containing a
> > > password assigned (you can find similar ones in your log-files as well)?
> > >
> > > 2017-11-28 10:19:27,981 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (API-Job-Executor-14:ctx-6858662d job-1158151 ctx-1967e9d7)
> > > (logid:eed0e79e) Complete async job-1158151, jobStatus: SUCCEEDED,
> > > resultCode: 0, result: org.apache.cloudstack.api.resp
> > > onse.UserVmResponse/virtualmachine/{"id":"57ec4f9a-9f65-
> > > 46c5-926d-a475bbe5c1d5","name":"VM-57ec4f9a-9f65-46c5-926d-a
> > > 475bbe5c1d5","displayname":"VM-57ec4f9a-9f65-46c5-926d-a475b
> > > be5c1d5","account":"admin","userid":"b11c5858-5357-497d-
> > > 93e7-f68db82535e7","username":"admin","domainid":"4d767ff4-
> > > 8216-4718-8f04-4626eeb5180f","domain":"2017102413000103","
> > > created":"2017-10-27T10:57:11+0300","state":"Stopped","
> > > haenable":false,"zoneid":"c8d773fa-76ca-4637-8ecf-
> > > 88656444fc86","zonename":"z2.tucha13.net","templateid":"
> > > 3b4b2504-9718-407e-8cf2-cdd286a90e52","templatename":"
> > > linux-ubuntu-desktop-16.04-x64-20170819","templatedisplaytext":"Linux
> > > Ubuntu 16.04 x64 Desktop version (rev.20170819)","passwordenabl
> > > ed":true,"serviceofferingid":"5248afa9-f896-4608-bf3b-
> > > 316262c21b9d","serviceofferingname":"custom-ssd-a1","
> > > cpunumber":1,"cpuspeed":2399,"memory":1024,"cpuused":"0.07%"
> > > ,"networkkbsread":417369,"networkkbswrite":58495,"diskkbsrea
> > > d":360776,"diskkbswrite":1978872,"memorykbs":1048576,"m
> > > emoryintfreekbs":1112364,"memorytargetkbs":1048576,"diskiore
> > > ad":11950,"diskiowrite":149126,"guestosid":"ca0edf48-
> > > bd31-11e6-b74f-06973a00088a","rootdeviceid":0,"rootdevicetyp
> > > e":"ROOT","securitygroup":[],"password":"*","
> > > nic":[{"id":"677447a3-de67-4477-b3fc-213ab12bf0d6","
> > > networkid":"1093f687-0581-4c63-9077-1471a8bfe7fd","
> > > networkname":"NET-PUB-193.151.666.666-24","netmask":"255.
> > > 255.255.0","gateway":"193.151.666.666","ipaddress":"193.151.
> > > 666.666","isolationuri":"vlan://100","broadcasturi":"vlan://
> > > 100","traffictype":"Guest","type":"Shared","isdefault":
> > > true,"macaddress":"66:66:66:66:66:66","secondaryip":[]},{"
> > > id":"3f71910e-cfe5-4d61-b725-e78e1d434cd8","networkid":"3422
> > > bda5-f206-4418-8a8a-30372a4f1e4a","networkname":"NET-
> > > 2017102413000103","netmask":"255.255.255.0","gateway":"192.
> > > 168.131.254","ipaddress":"192.168.131.154","traf
Re: Where is the vm root password published?
Aye, should be cool to have them encrypted by some RSA-key that would be
installed to the VM's template.
Though at this moment one should keep an eye on the systems where these logs
are stored.
On Tue, Nov 28, 2017 at 03:39:55PM +0530, Makrand wrote:
> Assuming all the passwords appearing in logs must be masked (kind of
> encrypted) How does one decrypt those password from logs?
>
> BTW, if passwords are just logged as plain text (even for temp amount of
> time), or stored as plain text over VR, then that's not a very secure
> thing, is it??
>
> --
> Makrand
>
>
> On Tue, Nov 28, 2017 at 2:58 PM, Vladimir Melnik <v.mel...@uplink.ua> wrote:
>
> > Hello,
> >
> > Would you mind if I share a sample line from the log-file containing a
> > password assigned (you can find similar ones in your log-files as well)?
> >
> > 2017-11-28 10:19:27,981 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (API-Job-Executor-14:ctx-6858662d job-1158151 ctx-1967e9d7)
> > (logid:eed0e79e) Complete async job-1158151, jobStatus: SUCCEEDED,
> > resultCode: 0, result: org.apache.cloudstack.api.resp
> > onse.UserVmResponse/virtualmachine/{"id":"57ec4f9a-9f65-
> > 46c5-926d-a475bbe5c1d5","name":"VM-57ec4f9a-9f65-46c5-926d-a
> > 475bbe5c1d5","displayname":"VM-57ec4f9a-9f65-46c5-926d-a475b
> > be5c1d5","account":"admin","userid":"b11c5858-5357-497d-
> > 93e7-f68db82535e7","username":"admin","domainid":"4d767ff4-
> > 8216-4718-8f04-4626eeb5180f","domain":"2017102413000103","
> > created":"2017-10-27T10:57:11+0300","state":"Stopped","
> > haenable":false,"zoneid":"c8d773fa-76ca-4637-8ecf-
> > 88656444fc86","zonename":"z2.tucha13.net","templateid":"
> > 3b4b2504-9718-407e-8cf2-cdd286a90e52","templatename":"
> > linux-ubuntu-desktop-16.04-x64-20170819","templatedisplaytext":"Linux
> > Ubuntu 16.04 x64 Desktop version (rev.20170819)","passwordenabl
> > ed":true,"serviceofferingid":"5248afa9-f896-4608-bf3b-
> > 316262c21b9d","serviceofferingname":"custom-ssd-a1","
> > cpunumber":1,"cpuspeed":2399,"memory":1024,"cpuused":"0.07%"
> > ,"networkkbsread":417369,"networkkbswrite":58495,"diskkbsrea
> > d":360776,"diskkbswrite":1978872,"memorykbs":1048576,"m
> > emoryintfreekbs":1112364,"memorytargetkbs":1048576,"diskiore
> > ad":11950,"diskiowrite":149126,"guestosid":"ca0edf48-
> > bd31-11e6-b74f-06973a00088a","rootdeviceid":0,"rootdevicetyp
> > e":"ROOT","securitygroup":[],"password":"*","
> > nic":[{"id":"677447a3-de67-4477-b3fc-213ab12bf0d6","
> > networkid":"1093f687-0581-4c63-9077-1471a8bfe7fd","
> > networkname":"NET-PUB-193.151.666.666-24","netmask":"255.
> > 255.255.0","gateway":"193.151.666.666","ipaddress":"193.151.
> > 666.666","isolationuri":"vlan://100","broadcasturi":"vlan://
> > 100","traffictype":"Guest","type":"Shared","isdefault":
> > true,"macaddress":"66:66:66:66:66:66","secondaryip":[]},{"
> > id":"3f71910e-cfe5-4d61-b725-e78e1d434cd8","networkid":"3422
> > bda5-f206-4418-8a8a-30372a4f1e4a","networkname":"NET-
> > 2017102413000103","netmask":"255.255.255.0","gateway":"192.
> > 168.131.254","ipaddress":"192.168.131.154","traffictype":"
> > Guest","type":"Isolated","isdefault":false,"macaddress":
> > "66:66:66:66:66:66","secondaryip":[]}],"hypervisor"
> > :"KVM","instancename":"i---VM","affinitygroup":[],"d
> > isplayvm":true,"isdynamicallyscalable":false,"ostypeid":254,"tags":[]}
> >
> > ^^^ That doesn't seem to be cloudmonkey who adds that to the management
> > log-file, as we don't use it at all.
> >
> > But there's a dilemma that needs to be solved, as "fixing&qu
Re: Where is the vm root password published?
Hello,
Would you mind if I share a sample line from the log-file containing a password
assigned (you can find similar ones in your log-files as well)?
2017-11-28 10:19:27,981 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-14:ctx-6858662d job-1158151 ctx-1967e9d7) (logid:eed0e79e)
Complete async job-1158151, jobStatus: SUCCEEDED, resultCode: 0, result:
org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"57ec4f9a-9f65-46c5-926d-a475bbe5c1d5","name":"VM-57ec4f9a-9f65-46c5-926d-a475bbe5c1d5","displayname":"VM-57ec4f9a-9f65-46c5-926d-a475bbe5c1d5","account":"admin","userid":"b11c5858-5357-497d-93e7-f68db82535e7","username":"admin","domainid":"4d767ff4-8216-4718-8f04-4626eeb5180f","domain":"2017102413000103","created":"2017-10-27T10:57:11+0300","state":"Stopped","haenable":false,"zoneid":"c8d773fa-76ca-4637-8ecf-88656444fc86","zonename":"z2.tucha13.net","templateid":"3b4b2504-9718-407e-8cf2-cdd286a90e52","templatename":"linux-ubuntu-desktop-16.04-x64-20170819","templatedisplaytext":"Linux
Ubuntu 16.04 x64 Desktop version
(rev.20170819)","passwordenabled":true,"serviceofferingid":"5248afa9-f896-4608-bf3b-316262c21b9d","serviceofferingname":"custom-ssd-a1","cpunumber":1,"cpuspeed":2399,"memory":1024,"cpuused":"0.07%","networkkbsread":417369,"networkkbswrite":58495,"diskkbsread":360776,"diskkbswrite":1978872,"memorykbs":1048576,"memoryintfreekbs":1112364,"memorytargetkbs":1048576,"diskioread":11950,"diskiowrite":149126,"guestosid":"ca0edf48-bd31-11e6-b74f-06973a00088a","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[],"password":"*","nic":[{"id":"677447a3-de67-4477-b3fc-213ab12bf0d6","networkid":"1093f687-0581-4c63-9077-1471a8bfe7fd","networkname":"NET-PUB-193.151.666.666-24","netmask":"255.255.255.0","gateway":"193.151.666.666","ipaddress":"193.151.666.666","isolationuri":"vlan://100","broadcasturi":"vlan://100","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"66:66:66:66:66:66","secondaryip":[]},{"id":"3f71910e-cfe5-4d61-b725-e78e1d434cd8","networkid":"3422bda5-f206-4418-8a8a-30372a4f1e4a","networkname":"NET-2017102413000103","netmask":"255.255.255.0","gateway":"192.168.131.254","ipaddress":"192.168.131.154","traffictype":"Guest","type":"Isolated","isdefault":false,"macaddress":"66:66:66:66:66:66","secondaryip":[]}],"hypervisor":"KVM","instancename":"i---VM","affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":254,"tags":[]}
^^^ That doesn't seem to be cloudmonkey who adds that to the management
log-file, as we don't use it at all.
But there's a dilemma that needs to be solved, as "fixing" that would mean that
a content-neutral logging module should understand which information is
confidential and shouldn't been logged, not such an easy task to be solved
properly.
With best,
Vlad
On Mon, Nov 27, 2017 at 05:02:00PM -0200, Rafael Weingärtner wrote:
> Ah, thanks Daan ;)
>
> On Mon, Nov 27, 2017 at 4:27 PM, Daan Hoogland
> wrote:
>
> > it isn't logged, Rafael, not by cloudstack. It is cloudmonkey that logs the
> > API response object. It is the same response the UI uses to display it to
> > the user.
> >
> > On Mon, Nov 27, 2017 at 3:45 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > Interesting! I did not know that the password was logged. I thought it
> > was
> > > a one time thing to show the password in the UI.
> > >
> > > On Mon, Nov 27, 2017 at 1:43 PM, Nux! wrote:
> > >
> > > > Ok, so found out some more stuff.
> > > >
> > > > First of all, the password appears in management-server.log and
> > > > apilog.log, so that's one place to grep into.
> > > >
> > > > Second, I could query the jobid and get the password from there. E.g.
> > > from
> > > > cloudmonkey
> > > > query asyncjobresult jobid=caac0e1f-0aff-4065-8189-1d32d480e73f | grep
> > > > password\ =
> > > >
> > > > More info here
> > > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/
> > > > CloudStack+cloudmonkey+CLI#CloudStackcloudmonkeyCLI-AsyncJobexecution
> > > >
> > > > --
> > > > Sent from the Delta quadrant using Borg technology!
> > > >
> > > > Nux!
> > > > www.nux.ro
> > > >
> > > > - Original Message -
> > > > > From: "Rafael Weingärtner"
> > > > > To: "users"
> > > > > Sent: Monday, 27 November, 2017 15:21:30
> > > > > Subject: Re: Where is the vm root password published?
> > > >
> > > > > Ah, if that is the case, I know it is stored in the VR of the network
> > > > where
> > > > > the VM is connected to.
> > > > >
> > > > > I forgot now the file, but it is something like
> > “/var/usr?/cloud/cache”
> > > > or
> > > > > something that ends in “/cache/cloud”.
> > > > >
> > > > >
> > > > > Do we store these password in ACS database as well?
> > > > >
> > > > > On Mon, Nov 27, 2017 at 1:18 PM, Nux! wrote:
> > > > >
> > > > >> Rafael,
> > > > >>
> > > > >> Yes
ACS attempts to re-download a template/ISO
Dear colleagues, Did anyone experienced attempts to download a template/ISO that had been already loaded some time ago. In my case it happens after the management-server restart (running 4.9.2). Have to update these templates' info in the `template_store_ref` table each time it happens (path = 'template/tmpl/13/666/13-666-01234567-89ab-cdef-fedc-ba9876543210.iso', download_state = 'DOWNLOADED', error_str = 'Install completed successfully at 13/13/13 04:20 AM', state= 'Ready'), but would like to prevent that. So, any tiny piece of advice would be appreciated. -- V.Melnik
LXC support (limits)
Dear colleagues, Had anyone succeeded at limiting an LXC-instance that has been deployed by ACS? On a CentOS-7-based host the VM can use all the CPU cores and all the storage. Is that possible to limit at least the storage usage? Any hints would be greatly appreciated. Thank you! -- V.Melnik
LXC support (the System VM template)
Dear colleagues, I have a couple of questions regarding LXC support if anyone doesn't mind. The first question seems to be really simple, but I'd like to make sure that I'm on the right way. I've added an LXC-zone to the ACS setup where I already have a KVM-zone. OK. I've added a host, but now I can't find the System VM template installed. I see this template in the list of templates and it even said that its' download is completed (https://www.dropbox.com/s/pm03jcuvipxtnmk/Screenshot%202017-05-11%2014.47.41.png?dl=0), but there are no files in my secondary storage which I've set up for the new zone. The directory is empty, so, when I'm enabling the zone, the cloudstack-agent's log shows the following: --- 8< --- 2017-05-10 22:53:05,857 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:c98f90c7) Attempting to create storage poo l 974cb6d1-7064-3a53-9d37-698037e367d0 (NetworkFilesystem) in libvirt 2017-05-10 22:53:05,858 WARN [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:c98f90c7) Storage pool 974cb6d1-7064-3a53- 9d37-698037e367d0 was not found running in libvirt. Need to create it. 2017-05-10 22:53:05,859 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:c98f90c7) Didn't find an existing storage pool 974cb6d1-7064-3a53-9d37-698037e367d0 by UUID, checking for pools with duplicate paths 2017-05-10 22:53:05,891 ERROR [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:c98f90c7) org.libvirt.LibvirtException: in ternal error Child process (/bin/mount 172.22.39.224:/export/secondary3/template/tmpl/1/10 /mnt/974cb6d1-7064-3a53-9d37-698037e367d0) unexpected exit status 32: mount.nfs: mounting 172.22.39.224:/export/secondary3/template/tmpl/1/10 failed, reason given by server: No such file or directory 2017-05-10 22:53:05,892 ERROR [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:c98f90c7) Internal error occurred when att empting to mount: specified path may be invalid 2017-05-10 22:53:05,892 ERROR [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:c98f90c7) Failed to create netfs mount: 17 2.22.39.224:/export/secondary3/template/tmpl/1/10 org.libvirt.LibvirtException: internal error Child process (/bin/mount 172.22.39.224:/export/secondary3/template/tmpl/1/10 /mnt/974cb6d1-7064-3a53-9d37-698037e367d0) unexpected exit status 32: mount.nfs: mounting 172.22.39.224:/export/secondary3/template/tmpl/1/10 failed, reason given by server: No such file or directory --- >8 --- In the same time I see the following records in the database: --- 8< --- mysql> SELECT * FROM vm_template WHERE id = 10; ++-+-+--++--++-+--+--++-+-++--+-+-+---+-+--+-+-+-+-++--+--+--++--+-+--+ | id | unique_name | name| uuid | public | featured | type | hvm | bits | url | format | created | removed | account_id | checksum | display_text | enable_password | enable_sshkey | guest_os_id | bootable | prepopulate | cross_zones | extractable | hypervisor_type | source_template_id | template_tag | sort_key | size | state | update_count | updated | dynamically_scalable | ++-+-+--++--++-+--+--++-+-++--+-+-+---+-+--+-+-+-+-++--+--+--++--+-+--+ | 10 | routing-10 | SystemVM Template (LXC) | a15d620e-bd31-11e6-b74f-06973a00088a | 0 |0 | SYSTEM | 0 | 64 | http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 | QCOW2 | 2016-12-08 12:32:33 | NULL| 1 | c059b0d051e0cd6fbe9d5d4fc40c7e5d | SystemVM Template (LXC) | 0 | 0 | 15 |1 | 0 | 1 | 0 | LXC | NULL | NULL |0 | NULL | Active |0 | NULL|0 |
Re: Using CentOS-6.x on KVM-hosts - what are the threats?
Thank you for the comment, Simon! The most funny thing is that I've added 3 new hosts to my infrastructure in December, their hardware is awesome, so their hardware refresh is not a matter of the nearest future. :) Anyhow, 6.x works great and the only thing I regret is lacking certain features (such as IOpS limits). I'm also maintaining a cluster of 5 hosts (primary storages aren't local, they're connected via GlusterFS & NFS) which've been running 6.x too. I've upgraded 3 of hosts from 6.x to 7.x, but when I'm trying to migrate a VM from the "old" hosts to the "new" ones, the migration is being timed out and the VM is being frozen in the "paused" state. I noticed a difference in the CPU-flags set: all the hosts running 7.x have the "nopl" flag, but the hosts running 6.x don't. This option appears only after installing 7.x and maybe this is the cause. Does anyone have any suggestions on the reason that causes freezing the VMs when they've been migrating from the 6.x-powered hosts to the 7.x-powered ones? Is that the "nopl" flag? Is that anything else? Thanks to all!
Using CentOS-6.x on KVM-hosts - what are the threats?
Dear colleagues, I've just realized that my KVM-hosts are running CentOS-6 whilst it's recommended to use CentOS-7 with the new versions of ACS. Everything seems to be fine (some of these hosts are working for a few years), hosts are working and things are great, but I'd like to ask a couple of questions. Here they are. (1) How high is the chance of the next version of ACS (4.10 or 4.11) will be incompatible with CentOS-6? Should I worry about that and consider upgrading to CentOS-7 immediately? (2) What ACS features I'm missing because of that? I suppose that I'll be disappointed if I try to limit a VM's IO-consumption, just because old good QEMU-0.9 won't support it. Am I right? Are there other things that are worth of upgraging to CentOS-7? Thank you very much in advance for your replies! -- V.Melnik
Re: Local LVM on KVM hosts
Hello, Thanks! I'm sorry, maybe I wasn't clear enough, but the thing I meant is using separate LVM volumes without QEMU-images. :) On Thu, Feb 16, 2017 at 01:54:54PM +, Dag Sonstebo wrote: > Hi Vladimir, > > Yes you can use local storage on KVM – you need to: > > 1) Enable it when you create the zone. > 2) Create service offerings with local storage. > > Regards, > Dag Sonstebo > Cloud Architect > ShapeBlue > > On 16/02/2017, 13:36, "Vladimir Melnik" <v.mel...@uplink.ua> wrote: > > Hello, > > Is there a way to use the local LVMs as a primary storage on the KVM > hosts? > > Thanks! > > -- > V.Melnik > > > > dag.sonst...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > -- V.Melnik
Local LVM on KVM hosts
Hello, Is there a way to use the local LVMs as a primary storage on the KVM hosts? Thanks! -- V.Melnik
Re: Enabling IPv6 in ACS-4.8
Thank you very much for the information. Does anyone use IPv6 in ACS at the moment? Is it working fine? On Thu, Feb 16, 2017 at 11:28:06AM +, Dag Sonstebo wrote: > Hi Vladimir, > > As you point out there doesn't’ seem to be a IPv6 specific system VM > template. I believe the guys at PCExtreme may have been working on this, I’ve > cheekily cc’ed them in and hopefully they can give you an update. > > Regards, > Dag Sonstebo > Cloud Architect > ShapeBlue > > On 14/02/2017, 16:15, "Vladimir Melnik" <v.mel...@uplink.ua> wrote: > > Hello, > > According to the documentation > (http://docs.cloudstack.apache.org/en/latest/networking/ipv6.html), it's > required to upgrade the SystemVM template to enable IPv6 support on virtual > routers. Where can I found the image? There's a reference to > http://cloudstack.apt-get.eu/systemvm/ in the manual, but > http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 > seems to be the same image that I've installed when I set up, so - do I > really need to upgrade it? > > Thanks! > > -- > V.Melnik > > > > dag.sonst...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > -- V.Melnik
Enabling IPv6 in ACS-4.8
Hello, According to the documentation (http://docs.cloudstack.apache.org/en/latest/networking/ipv6.html), it's required to upgrade the SystemVM template to enable IPv6 support on virtual routers. Where can I found the image? There's a reference to http://cloudstack.apt-get.eu/systemvm/ in the manual, but http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 seems to be the same image that I've installed when I set up, so - do I really need to upgrade it? Thanks! -- V.Melnik
Is there a way to assign or change an IP-address of a VR in a shared network?
Hello, Is there a way to assign an arbitrary address to a VR in a shared (guest) network? Thank you! -- V.Melnik
Re: Adding a secondary management server
Oh, yes, how could I forget the "host" parameter?? Thank you very much! On Sun, Dec 04, 2016 at 02:15:06AM +0530, Vivek Kumar wrote: > Hello Vlamdir, > > you need to change settings in global settings, Go to your global settings > in your cloudstack and search for "host", i m sure it will be your 1st > management ip( x.x.x.1) , so remove this out and put the virtual ip ( Load > balancer IP- x.x.x.3 ) in host section and then restart all cloudstack > services on both of yours management server. > > > *Vivek Kumar* > Virtualization and Cloud Consultant > > [image: http://www.indiqus.com/images/logo.jpg] <http://www.indiqus.com/> > *I*ndi*Q*us Technologies Pvt Ltd > A-98, LGF, C.R.Park, New Delhi - 110019 > *O* +91 11 4055 1411 | *M* +91 7503460090 > www.indiqus.com <http://www.indiqus.com/> > > On Sun, Dec 4, 2016 at 2:02 AM, Vladimir Melnik <v.mel...@uplink.ua> wrote: > > > Dear colleagues, > > > > Would you be so kind as to share your experience of running multiple > > servers? > > > > I have 2 management servers, they have different IP-addresses (x.x.x.1, > > x.x.x.2) and there are also a common IP-address (x.x.x.3) that is being > > provided by a load-balancer (haproxy). Also I have some hosts and some > > system VMs, they all connect to x.x.x.1, because x.x.x.1 was the > > management-server who was used to set-up the zone. > > > > It's easy to let the hosts know that they need to connect to x.x.x.3, as > > this parameter is being set in agent.properties, but where to change it in > > the system VMs? Grep haven't found x.x.x.1 anywhere but the log-files. > > Moreover, if I destroy and create the system VMs, they'll keep establishing > > connections to x.x.x.1. How to change their behaviour? > > > > Had I add "-m x.x.x.3" when I was running cloudstack-setup-database to > > prevent that? It's a new setup, so it's still okay to reset it. :) > > > > Thanks to all for sharing clues and hints! > > > > -- > > V.Melnik > > -- V.Melnik
Adding a secondary management server
Dear colleagues, Would you be so kind as to share your experience of running multiple servers? I have 2 management servers, they have different IP-addresses (x.x.x.1, x.x.x.2) and there are also a common IP-address (x.x.x.3) that is being provided by a load-balancer (haproxy). Also I have some hosts and some system VMs, they all connect to x.x.x.1, because x.x.x.1 was the management-server who was used to set-up the zone. It's easy to let the hosts know that they need to connect to x.x.x.3, as this parameter is being set in agent.properties, but where to change it in the system VMs? Grep haven't found x.x.x.1 anywhere but the log-files. Moreover, if I destroy and create the system VMs, they'll keep establishing connections to x.x.x.1. How to change their behaviour? Had I add "-m x.x.x.3" when I was running cloudstack-setup-database to prevent that? It's a new setup, so it's still okay to reset it. :) Thanks to all for sharing clues and hints! -- V.Melnik
Can I limit the number of CPUs per a VM?
Dear colleagues, Are there any ways to limit the number of CPUs per a VM? For example, I'd like not to allow my clients to run VMs that would have more than 64 CPUs, is there a way to have such a limit set? Thanks in advance for any clues! -- V.Melnik
Re: Storage Performance
Dear colleagues, I've found why guest's storage performance was much less than host's performance (the mistake was too stupid to tell about it, really). But I'd like to ask one more question if you don't mind. :) I played with various KVM options (cache, io and so on...) and now I can say that I got most IOpS with cache=writeback. Tests are being performed with the fio utility, here is the jobfile contents: [readtest] blocksize=4k filename=/dev/vdb rw=randread direct=1 buffered=0 ioengine=libaio iodepth=32 size=20% [writetest] blocksize=4k filename=/dev/vdb rw=randwrite direct=1 buffered=0 ioengine=libaio iodepth=32 size=20% Is there any way to make ACS run create KVM domains with cache=writeback instead of cache=none without patching the source code? Thank you! -- V.Melnik
Re: Storage Performance
I use CentOS-6.8 as the operating system of a host.
Thanks!
On Sun, Jun 05, 2016 at 07:05:30PM +0200, Timothy Lothering wrote:
> Hi Vladimir,
>
> What hypervisor are you using?
>
> -Original Message-
> From: Vladimir Melnik [mailto:v.mel...@uplink.ua]
> Sent: Sunday, 05 June 2016 6:06 PM
> To: users@cloudstack.apache.org
> Subject: Storage Performance
>
> Hello,
>
> I have an ACS-driven environment with a storage subsystem which is built on
> Gluster over InfiniBand. The storage shows pretty good performance when I
> mount a volume on a host and run a simple test ("dd if=/dev/zero
> of=/mnt/tmp/test.1G bs=1G count=1 conv=fdatasync"), it shows about 400 MB/s
> and that's okay. But when I deploy a virtual machine (I tried it with
> CentOS-6.8-x64 as a guest OS), I can't gain so good result from inside of a
> guest (it shows about 40 MB/s with the same simple test).
>
> What do you think, have I forgotten to do something important when I was
> seting this environment up?
>
> Thank you very much for sharing your ideas and clues!
>
> --
> V.Melnik
>
--
V.Melnik
Storage Performance
Hello,
I have an ACS-driven environment with a storage subsystem which is built on
Gluster over InfiniBand. The storage shows pretty good performance when I mount
a volume on a host and run a simple test ("dd if=/dev/zero of=/mnt/tmp/test.1G
bs=1G count=1 conv=fdatasync"), it shows about 400 MB/s and that's okay. But
when I deploy a virtual machine (I tried it with CentOS-6.8-x64 as a guest OS),
I can't gain so good result from inside of a guest (it shows about 40 MB/s with
the same simple test).
What do you think, have I forgotten to do something important when I was seting
this environment up?
Thank you very much for sharing your ideas and clues!
--
V.Melnik
Apache CloudStack meetups and conferences
Hello, Are there any ACS-related conferences scheduled to the beginning of May'16? What sources should I monitor to be aware of the meetups planned? Thank you! -- V.Melnik
RE: Apache CloudStack meetups and conferences
Hi! I live in Ukraine, but in the beginning of May I’ll be around London & Amsterdam in, so I’m going not to miss the meetup if there’ll be any. :) From: Giles Sirett [mailto:giles.sir...@shapeblue.com] Sent: Monday, December 21, 2015 1:22 PM To: users@cloudstack.apache.org Subject: RE: Apache CloudStack meetups and conferences Hi Vladimir Where abouts are you based ? Kind Regards Giles <http://www.shapeblue.com> Giles Sirett CEO , ShapeBlue d: <tel:+44%20203%20603%200540%20|%20s:%20+44%20%2020%203603%200541> +44 203 603 0540 | s: +44 20 3603 0541 | m: <tel:+44%207961112055> +44 7961112055 e: <mailto:giles.sir...@shapeblue.com%20|%20t:> giles.sir...@shapeblue.com | t: | w: <http://www.shapeblue.com> www.shapeblue.com a: 53 Chandos Place, Covent Garden London WC2N 4HS UK Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. -----Original Message- From: Vladimir Melnik [ <mailto:v.mel...@uplink.ua> mailto:v.mel...@uplink.ua] Sent: 21 December 2015 10:40 To: <mailto:users@cloudstack.apache.org> users@cloudstack.apache.org Subject: Apache CloudStack meetups and conferences Hello, Are there any ACS-related conferences scheduled to the beginning of May'16? What sources should I monitor to be aware of the meetups planned? Thank you! -- V.Melnik Find out more about ShapeBlue and our range of CloudStack related services: <http://shapeblue.com/iaas-cloud-design-and-build/> IaaS Cloud Design & Build | <http://shapeblue.com/csforge/> CSForge – rapid IaaS deployment framework <http://shapeblue.com/cloudstack-consultancy/> CloudStack Consulting | <http://shapeblue.com/cloudstack-software-engineering/> CloudStack Software Engineering <http://shapeblue.com/cloudstack-infrastructure-support/> CloudStack Infrastructure Support | <http://shapeblue.com/cloudstack-training/> CloudStack Bootcamp Training Courses
Ceph & KVM clusters on the same servers
Hello, Have anyone tried to use the same servers for a KVM virtualization HA-cluster and a Ceph storage cluster? So guests' VMs and volumes will be quite available. Of course, there'll be huge network load, so we'll have to design our network wisely. Maybe a separate network for the management VLAN and for the management/monitoring network of Ceph, a separate one for the secondary storage network, another one - for public & private VLANS of guests and, the last but not least, a network to interact with the Ceph storage from hypervisors. Should be working fine, shouldn't it? If anyone tried to implement this configuration, would you be so kind as to share any of your conclusions about its performance and reliability? Thank you very much in advance! -- V.Melnik
Re: The agent doesn't reconnect if there are stopped VMs
Hello, On Fri, Aug 21, 2015 at 04:41:42PM +, Somesh Naidu wrote: If you mean offerings with custom parameters - yes, I use them There is a possibility that this might be affecting this. Thank you! So, should I change all offerings to static ones? I don't have more information in the log-file Can you open the mgmt server log from 2015-08-18, search for 2015-08-18 06:24:46,544, the next few lines should contain the java exception stack which should point us to the method-file-line_number that will help us understand what is causing the NPE. I had posted the full log in the initial message. There is no exception stack. There are 3 lines: 1: 2015-08-18 06:24:46,544 ERROR [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-213:ctx-76903ef6) Monitor ComputeCapacityListener says there is an error in the connect process for 27 due to null 2: java.lang.NullPointerException 3: 2015-08-18 06:24:46,544 INFO [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-213:ctx-76903ef6) Host 27 is disconnecting with event AgentDisconnected The 2nd line doesn't contain an exception stack, all it contains is java.lang.NullPointerException. If there was an exception stack I probably wouldn't ask stupid questions here. :) -- V.Melnik P.S. I've opened an issue: https://issues.apache.org/jira/browse/CLOUDSTACK-8747
Re: The agent doesn't reconnect if there are stopped VMs
I'll gladly send it, but where to find it? I don't have more information in the
log-file :(
On Thu, Aug 20, 2015 at 03:49:25PM +, Somesh Naidu wrote:
2015-08-18 06:24:46,542 DEBUG [c.c.c.CapacityManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Found 1 VM, not running on host 27
2015-08-18 06:24:46,544 ERROR [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Monitor ComputeCapacityListener
says there is an error in the connect process for 27 due to null
java.lang.NullPointerException
It will help if you send the complete exception stack for the above NPE.
Regards,
Somesh
-Original Message-
From: Simon Weller [mailto:swel...@ena.com]
Sent: Thursday, August 20, 2015 11:21 AM
To: users@cloudstack.apache.org
Subject: Re: The agent doesn't reconnect if there are stopped VMs
Do you have any dynamic service offerings?
From: Vladimir Melnik v.mel...@uplink.ua
Sent: Thursday, August 20, 2015 8:20 AM
To: users@cloudstack.apache.org
Subject: Re: The agent doesn't reconnect if there are stopped VMs
Oh, I'm sorry, I should have initially send the DEBUG-log!
Here is an example:
--- 8 ---
2015-07-29 00:53:42,988 INFO [utils.nio.NioClient] (Agent-Selector:null)
Connecting to ***.***.***.***:8250
2015-07-29 00:53:44,254 INFO [utils.nio.NioClient] (Agent-Selector:null)
SSL: Handshake done
2015-07-29 00:53:44,255 INFO [utils.nio.NioClient] (Agent-Selector:null)
Connected to ***.***.***.***:8250
2015-07-29 00:53:44,258 WARN [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Could not read cpuinfo_max_freq
2015-07-29 00:53:44,266 DEBUG [kvm.resource.LibvirtCapXMLParser]
(Agent-Handler-1:null) Found /usr/libexec/qemu-kvm as a suiteable emulat
or
2015-07-29 00:53:44,266 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Executing: /bin/bash -c qemu-img --help|grep
convert
2015-07-29 00:53:44,270 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Execution is successful.
2015-07-29 00:53:44,271 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) convert [-c] [-p] [-f fmt] [-t cache] [-T
src_cache] [-O output_fmt] [-o options] [-S sparse_size] filename [filename2
[...]] output_filename
2015-07-29 00:53:44,271 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) cpus=8, speed=2660, ram=30153224192, dom0ram
=805306368, cpu sockets=1
2015-07-29 00:53:44,272 DEBUG [cloud.resource.ServerResourceBase]
(Agent-Handler-1:null) Parameters for private nic: 172.26.65.1 - 84:2b:
2b:56:d3:d9-255.255.255.0
2015-07-29 00:53:44,272 DEBUG [cloud.resource.ServerResourceBase]
(Agent-Handler-1:null) Parameters for storage nic: 172.26.65.1 - 84:2b:
2b:56:d3:d9-255.255.255.0
2015-07-29 00:53:44,272 DEBUG [cloud.resource.ServerResourceBase]
(Agent-Handler-1:null) Parameters for pubic nic: 172.26.65.1 - 84:2b:2b
:56:d3:d9-255.255.255.0
2015-07-29 00:53:44,272 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Executing: /usr/share/cloudstack-common/scri
pts/vm/hypervisor/versions.sh
2015-07-29 00:53:44,281 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Execution is successful.
2015-07-29 00:53:44,282 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Executing: sudo grep InitiatorName= /etc/isc
si/initiatorname.iscsi
2015-07-29 00:53:44,290 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Execution is successful.
2015-07-29 00:53:44,290 INFO [kvm.storage.LibvirtStorageAdaptor]
(Agent-Handler-1:null) Attempting to create storage pool 42bed7d9-88ae-
403d-9b53-0b44f31b2312 (Filesystem) in libvirt
2015-07-29 00:53:44,292 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(Agent-Handler-1:null) Found existing defined storage pool 42bed7d9-88a
e-403d-9b53-0b44f31b2312, using it.
2015-07-29 00:53:44,292 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(Agent-Handler-1:null) Trying to fetch storage pool
42bed7d9-88ae-403d-9b53-0b44f31b2312 from libvirt
2015-07-29 00:53:44,811 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
Executing: hostname
2015-07-29 00:53:44,813 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
Execution is successful.
2015-07-29 00:53:44,834 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
Executing: hostname
2015-07-29 00:53:44,836 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
Execution is successful.
2015-07-29 00:53:44,838 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
Sending Startup: Seq 0-64: { Cmd , MgmtId: -1, via: 0, Ver: v1, Flags: 1,
[{com.cloud.agent.api.StartupRoutingCommand:{cpuSockets:1,cpus:8,speed:2660,memory:30153224192,dom0MinMemory:805306368,poolSync:false,caps:hvm,snapshot,pool:/root,hypervisorType:KVM,hostDetails:{com.cloud.network.Networks.RouterPrivateIpStrategy:HostLocal,Host.OS:CentOS,Host.OS.Kernel.Version:2.6.32-504.16.2.el6.x86_64,Host.OS.Version
Re: The agent doesn't reconnect if there are stopped VMs
If you mean dynamic scaleable offerings - no I don't have any.
If you mean offerings with custom parameters - yes, I use them.
Thanks!
On Thu, Aug 20, 2015 at 03:21:14PM +, Simon Weller wrote:
Do you have any dynamic service offerings?
From: Vladimir Melnik v.mel...@uplink.ua
Sent: Thursday, August 20, 2015 8:20 AM
To: users@cloudstack.apache.org
Subject: Re: The agent doesn't reconnect if there are stopped VMs
Oh, I'm sorry, I should have initially send the DEBUG-log!
Here is an example:
--- 8 ---
2015-07-29 00:53:42,988 INFO [utils.nio.NioClient] (Agent-Selector:null)
Connecting to ***.***.***.***:8250
2015-07-29 00:53:44,254 INFO [utils.nio.NioClient] (Agent-Selector:null)
SSL: Handshake done
2015-07-29 00:53:44,255 INFO [utils.nio.NioClient] (Agent-Selector:null)
Connected to ***.***.***.***:8250
2015-07-29 00:53:44,258 WARN [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Could not read cpuinfo_max_freq
2015-07-29 00:53:44,266 DEBUG [kvm.resource.LibvirtCapXMLParser]
(Agent-Handler-1:null) Found /usr/libexec/qemu-kvm as a suiteable emulat
or
2015-07-29 00:53:44,266 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Executing: /bin/bash -c qemu-img --help|grep
convert
2015-07-29 00:53:44,270 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Execution is successful.
2015-07-29 00:53:44,271 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) convert [-c] [-p] [-f fmt] [-t cache] [-T
src_cache] [-O output_fmt] [-o options] [-S sparse_size] filename [filename2
[...]] output_filename
2015-07-29 00:53:44,271 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) cpus=8, speed=2660, ram=30153224192, dom0ram
=805306368, cpu sockets=1
2015-07-29 00:53:44,272 DEBUG [cloud.resource.ServerResourceBase]
(Agent-Handler-1:null) Parameters for private nic: 172.26.65.1 - 84:2b:
2b:56:d3:d9-255.255.255.0
2015-07-29 00:53:44,272 DEBUG [cloud.resource.ServerResourceBase]
(Agent-Handler-1:null) Parameters for storage nic: 172.26.65.1 - 84:2b:
2b:56:d3:d9-255.255.255.0
2015-07-29 00:53:44,272 DEBUG [cloud.resource.ServerResourceBase]
(Agent-Handler-1:null) Parameters for pubic nic: 172.26.65.1 - 84:2b:2b
:56:d3:d9-255.255.255.0
2015-07-29 00:53:44,272 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Executing: /usr/share/cloudstack-common/scri
pts/vm/hypervisor/versions.sh
2015-07-29 00:53:44,281 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Execution is successful.
2015-07-29 00:53:44,282 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Executing: sudo grep InitiatorName= /etc/isc
si/initiatorname.iscsi
2015-07-29 00:53:44,290 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-1:null) Execution is successful.
2015-07-29 00:53:44,290 INFO [kvm.storage.LibvirtStorageAdaptor]
(Agent-Handler-1:null) Attempting to create storage pool 42bed7d9-88ae-
403d-9b53-0b44f31b2312 (Filesystem) in libvirt
2015-07-29 00:53:44,292 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(Agent-Handler-1:null) Found existing defined storage pool 42bed7d9-88a
e-403d-9b53-0b44f31b2312, using it.
2015-07-29 00:53:44,292 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(Agent-Handler-1:null) Trying to fetch storage pool
42bed7d9-88ae-403d-9b53-0b44f31b2312 from libvirt
2015-07-29 00:53:44,811 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
Executing: hostname
2015-07-29 00:53:44,813 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
Execution is successful.
2015-07-29 00:53:44,834 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
Executing: hostname
2015-07-29 00:53:44,836 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
Execution is successful.
2015-07-29 00:53:44,838 DEBUG [cloud.agent.Agent] (Agent-Handler-1:null)
Sending Startup: Seq 0-64: { Cmd , MgmtId: -1, via: 0, Ver: v1, Flags: 1,
[{com.cloud.agent.api.StartupRoutingCommand:{cpuSockets:1,cpus:8,speed:2660,memory:30153224192,dom0MinMemory:805306368,poolSync:false,caps:hvm,snapshot,pool:/root,hypervisorType:KVM,hostDetails:{com.cloud.network.Networks.RouterPrivateIpStrategy:HostLocal,Host.OS:CentOS,Host.OS.Kernel.Version:2.6.32-504.16.2.el6.x86_64,Host.OS.Version:6.6},hostTags:[],groupDetails:{},type:Routing,dataCenter:4,pod:5,cluster:5,guid:26e2bf7d-2fcf-3a67-a23d-ce9c09ef2ca5-LibvirtComputingResource,name:***.***.***,id:0,version:4.5.1,iqn:iqn.1994-05.com.redhat:f044a5e741a1,publicIpAddress:172.26.65.1,publicNetmask:255.255.255.0,publicMacAddress:84:2b:2b:56:d3:d9,privateIpAddress:172.26.65.1,privateMacAddress:84:2b:2b:56:d3:d9,privateNetmask:255.255.255.0,storageIpAddress:172.26.65.1,storageNetmask:255.255.255.0,storageMacAddress:84:2b:2b:56:d3:d9,resourceName:LibvirtComputingResource,gatewayIpAddress:103.247.149.1,wait:0}},{com.cloud.agent.api.StartupStorageCommand:{totalSize:0,poolInfo:{uuid
The agent doesn't reconnect if there are stopped VMs
Dear colleagues,
I have a simple setup where the management server (CentOS-6.6 +
ACS-4.5.1) is orchestrating a bunch of KVM hosts (each of them is
running CentOS-6.6 + ACS-4.5.1 as well).
Any host with at least one VM in the Stopped state can't reconnect to
the management server. It has the Alert state and here's what I see in
the management server's log-file:
--- 8 ---
2015-08-18 06:24:46,332 DEBUG [c.c.a.t.Request]
(AgentConnectTaskPool-213:ctx-76903ef6) Seq 0-148: Processing the first command
{ Cmd ,
MgmtId: -1, via: 0, Ver: v1, Flags: 1,
[{com.cloud.agent.api.StartupRoutingCommand:{cpuSockets:1,cpus:48,speed:2299,memory:6743
9632384,dom0MinMemory:805306368,poolSync:false,caps:hvm,snapshot,pool:/root,hypervisorType:KVM,hostDetails:{com.cloud.
network.Networks.RouterPrivateIpStrategy:HostLocal,Host.OS:CentOS,Host.OS.Kernel.Version:2.6.32-504.23.4.el6.x86_64,Host.OS.V
ersion:6.6},hostTags:[],groupDetails:{},type:Routing,dataCenter:6,pod:7,cluster:7,guid:1318c38d-4ed6-3296-a6bd-753676e25ad4-LibvirtComputingResource,name:***.***.***,id:0,version:4.5.1,publicIpAddress:172.27.65.1,publicNetmask:255.255.255.0,publicMacAddress:ec:f4:bb:d6:89:c5,privateIpAddress:172.27.65.1,privateMacAddress:ec:f4:bb:d6:89:c5,privateNetmask:255.255.255.0,storageIpAddress:172.27.65.1,storageNetmask:255.255.255.0,storageMacAddress:ec:f4:bb:d6:89:c5,resourceName:LibvirtComputingResource,gatewayIpAddress:***.***.***.***,wait:0}},{com.cloud.agent.api.StartupStorageCommand:{totalSize:0,poolInfo:{uuid:51670fbd-ece2-4a3e-9971-3928e6576f0e,host:172.27.65.1,localPath:/var/lib/libvirt/images,hostPath:/var/lib/libvirt/images,poolType:Filesystem,capacityBytes:1563804868608,availableBytes:1474368700416},resourceType:STORAGE_POOL,hostDetails:{},type:Storage,dataCenter:6,pod:7,guid:1318c38d-4ed6-3296-a6bd-753676e25ad4-LibvirtComputingResource,name:***.***.***,id:0,version:4.5.1,resourceName:LibvirtComputingResource,wait:0}}]
}
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to BaremetalDhcpManagerImpl
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to BaremetalPxeManagerImpl
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to NetworkUsageManagerImpl
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to NuageVspElement
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to Ovs
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to PaloAltoExternalFirewallElement
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to GloboDnsElement
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to KvmServerDiscoverer
2015-08-18 06:24:46,362 DEBUG [c.c.r.ResourceState]
(AgentConnectTaskPool-213:ctx-76903ef6) Resource state update: [id = 27; name =
***.***.***; old state = Enabled; event = InternalCreated; new state = Enabled]
2015-08-18 06:24:46,362 DEBUG [c.c.h.Status]
(AgentConnectTaskPool-213:ctx-76903ef6) Transition:[Resource state = Enabled,
Agent event = AgentConnected, Host id = 27, name = ***.***.***]
2015-08-18 06:24:46,365 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) create ClusteredAgentAttache for 27
2015-08-18 06:24:46,367 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Sending Connect to listener:
XcpServerDiscoverer
2015-08-18 06:24:46,367 DEBUG [c.c.h.x.d.XcpServerDiscoverer]
(AgentConnectTaskPool-213:ctx-76903ef6) Not XenServer so moving on.
2015-08-18 06:24:46,367 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Sending Connect to listener:
HypervServerDiscoverer
2015-08-18 06:24:46,367 DEBUG [c.c.h.h.d.HypervServerDiscoverer]
(AgentConnectTaskPool-213:ctx-76903ef6) Not Hyper-V hypervisor, so moving on.
2015-08-18 06:24:46,367 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Sending Connect to listener:
ClusteredVirtualMachineManagerImpl
2015-08-18 06:24:46,367 DEBUG [c.c.v.VirtualMachineManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Received startup command from
hypervisor host. host id: 27
2015-08-18 06:24:46,367 INFO [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Reset VM power state sync
Re: The agent doesn't reconnect if there are stopped VMs
:53:46,196 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null)
Request:Seq 22-7511722703477276676: { Cmd , MgmtId: 279278805451086, via: 22,
Ver: v1, Flags: 100011,
[{com.cloud.agent.api.ModifySshKeysCommand:{wait:0}}] }
2015-07-29 00:53:46,196 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null)
Processing command: com.cloud.agent.api.ModifySshKeysCommand
2015-07-29 00:53:46,197 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Executing: chmod 600 /root/.ssh/id_rsa.cloud
2015-07-29 00:53:46,198 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Execution is successful.
2015-07-29 00:53:46,199 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null)
Seq 22-7511722703477276676: { Ans: , MgmtId: 279278805451086, via: 22, Ver:
v1, Flags: 10, [{com.cloud.agent.api.Answer:{result:true,wait:0}}] }
2015-07-29 00:53:46,199 WARN [cloud.agent.Agent] (agentRequest-Handler-2:null)
Unable to send response: Seq 22-7511722703477276676: { Ans: , MgmtId:
279278805451086, via: 22, Ver: v1, Flags: 10,
[{com.cloud.agent.api.Answer:{result:true,wait:0}}] }
2015-07-29 00:53:49,255 INFO [cloud.agent.Agent] (Agent-Handler-4:null)
Connected to the server
2015-07-29 00:53:51,196 INFO [cloud.agent.Agent] (Agent-Handler-4:null) Lost
connection to the server. Dealing with the remaining commands...
2015-07-29 00:53:56,198 INFO [utils.nio.NioClient] (Agent-Handler-4:null)
NioClient connection closed
--- 8 ---
It seems that the connection is being closed by the management-server,
but I don't see why.
On Thu, Aug 20, 2015 at 12:53:33PM +, Simon Weller wrote:
Vladimir,
Could you turn up debugging on the agent and post another agent log?
You can do this by running: sed -i 's/INFO/DEBUG/g'
/etc/cloudstack/agent/log4j-cloud.xml
Then restart the agent.
- Si
From: Vladimir Melnik v.mel...@uplink.ua
Sent: Thursday, August 20, 2015 4:36 AM
To: users@cloudstack.apache.org
Subject: The agent doesn't reconnect if there are stopped VMs
Dear colleagues,
I have a simple setup where the management server (CentOS-6.6 +
ACS-4.5.1) is orchestrating a bunch of KVM hosts (each of them is
running CentOS-6.6 + ACS-4.5.1 as well).
Any host with at least one VM in the Stopped state can't reconnect to
the management server. It has the Alert state and here's what I see in
the management server's log-file:
--- 8 ---
2015-08-18 06:24:46,332 DEBUG [c.c.a.t.Request]
(AgentConnectTaskPool-213:ctx-76903ef6) Seq 0-148: Processing the first
command { Cmd ,
MgmtId: -1, via: 0, Ver: v1, Flags: 1,
[{com.cloud.agent.api.StartupRoutingCommand:{cpuSockets:1,cpus:48,speed:2299,memory:6743
9632384,dom0MinMemory:805306368,poolSync:false,caps:hvm,snapshot,pool:/root,hypervisorType:KVM,hostDetails:{com.cloud.
network.Networks.RouterPrivateIpStrategy:HostLocal,Host.OS:CentOS,Host.OS.Kernel.Version:2.6.32-504.23.4.el6.x86_64,Host.OS.V
ersion:6.6},hostTags:[],groupDetails:{},type:Routing,dataCenter:6,pod:7,cluster:7,guid:1318c38d-4ed6-3296-a6bd-753676e25ad4-LibvirtComputingResource,name:***.***.***,id:0,version:4.5.1,publicIpAddress:172.27.65.1,publicNetmask:255.255.255.0,publicMacAddress:ec:f4:bb:d6:89:c5,privateIpAddress:172.27.65.1,privateMacAddress:ec:f4:bb:d6:89:c5,privateNetmask:255.255.255.0,storageIpAddress:172.27.65.1,storageNetmask:255.255.255.0,storageMacAddress:ec:f4:bb:d6:89:c5,resourceName:LibvirtComputingResource,gatewayIpAddress:***.***.***.***,wait:0}},{com.cloud.agent.api.StartupStorageCommand:{totalSize:0,poolInfo:{uuid:51670fbd-ece2-4a3e-9971-3928e6576f0e,host:172.27.65.1,localPath:/var/lib/libvirt/images,hostPath:/var/lib/libvirt/images,poolType:Filesystem,capacityBytes:1563804868608,availableBytes:1474368700416},resourceType:STORAGE_POOL,hostDetails:{},type:Storage,dataCenter:6,pod:7,guid:1318c38d-4ed6-3296-a6bd-753676e25ad4-LibvirtComputingResource,name:***.***.***,id:0,version:4.5.1,resourceName:LibvirtComputingResource,wait:0}}]
}
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to BaremetalDhcpManagerImpl
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to BaremetalPxeManagerImpl
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to NetworkUsageManagerImpl
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to NuageVspElement
2015-08-18 06:24:46,336 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-213:ctx-76903ef6) Dispatching resource state event
CREATE_HOST_VO_FOR_CONNECTED to Ovs
2015-08-18 06:24:46,336 DEBUG
RE: The agent doesn't restart :(
Ah, now I see what's going on. The agent starts, but it doesn't work at all!
2015-05-12 18:12:49,493 INFO [cloud.agent.AgentShell] (Thread-1:null) Agent
started
2015-05-12 18:12:49,495 INFO [cloud.agent.AgentShell] (Thread-1:null)
Implementation Version is 4.2.1
2015-05-12 18:12:49,496 INFO [cloud.agent.AgentShell] (Thread-1:null)
agent.properties found at /etc/cloudstack/agent/agent.properties
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: workers
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: port
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: pod
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: resource
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: private.network.device
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: zone
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: guid
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: guest.network.device
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: cluster
2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: domr.scripts.dir
2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: local.storage.uuid
2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: public.network.device
2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: host
2015-05-12 18:12:49,499 INFO [cloud.agent.AgentShell] (Thread-1:null)
Defaulting to using properties file for storage
2015-05-12 18:12:49,500 INFO [cloud.agent.AgentShell] (Thread-1:null)
Defaulting to the constant time backoff algorithm
2015-05-12 18:12:49,501 INFO [cloud.utils.LogUtils] (Thread-1:null) log4j
configuration found at /etc/cloudstack/agent/log4j-cloud.xml
2015-05-12 18:12:49,614 DEBUG [cloud.agent.AgentShell] (Thread-1:null)
Checking to see if agent.pid exists.
2015-05-12 18:12:49,622 DEBUG [cloud.utils.ProcessUtil] (Thread-1:null)
Executing: bash -c echo $PPID
And this is the last line I see in the log-file. The file /var/log/agent.pid
is being created, but it has zero length. So, the agent's process is still
working (I can see it by ps), but it doesn't do anything.
Any ideas on the cause of such an odd behavior?
-Original Message-
From: Vladimir Melnik [mailto:v.mel...@uplink.ua]
Sent: Tuesday, May 12, 2015 5:57 PM
To: users@cloudstack.apache.org
Subject: The agent doesn't restart :(
Hello!
I encountered quite an odd problem: the agent doesn't restart on KVM host.
Here is what's going on.
When it starts for the first time it creates /var/run/agent.pid file, but
there are no numbers, the file has zero length.
When I restart it (e.g. by service cloudstack-agent restart command), it's
being shut down, but file isn't being removed. So it can't start again, here
is what I see in /var/log/cloudstack/agent/cloudstack-agent.out file:
2015-05-12 17:04:37,504{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
Agent started
2015-05-12 17:04:37,506{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
Implementation Version is 4.2.1
2015-05-12 17:04:37,507{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
agent.properties found at /etc/cloudstack/agent/agent.properties
2015-05-12 17:04:37,508{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
Defaulting to using properties file for storage
2015-05-12 17:04:37,509{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
Defaulting to the constant time backoff algorithm
2015-05-12 17:04:37,510{GMT} INFO [cloud.utils.LogUtils] (Thread-1:) log4j
configuration found at /etc/cloudstack/agent/log4j-cloud.xml
2015-05-12 17:04:37,626{GMT} ERROR [cloud.agent.AgentShell] (Thread-1:)
Unable to start agent: Java process is being started twice. If this is not
true, remove /var/run/agent.pid Unable to start agent: Java process is being
started twice. If this is not true, remove /var/run/agent.pid
I remove this file by hands and do restart again. The agent starts, but
/var/run/agent.pid has zero length again.
This server is just the same as other CentOS hosts (there are many of them
in my farm), but other hosts don't seem to be having this issue.
I opened a ticket (https://issues.apache.org/jira/browse/CLOUDSTACK-8456,
there are more details about the environment), but if anyone has some clues,
hints or ideas, please, share your thoughts on this topic. Any help will be
greatly appreciated! Thanks!
--
V.Melnik
The agent doesn't restart :(
Hello!
I encountered quite an odd problem: the agent doesn't restart on KVM host.
Here is what's going on.
When it starts for the first time it creates /var/run/agent.pid file, but there
are no numbers, the file has zero length.
When I restart it (e.g. by service cloudstack-agent restart command), it's
being shut down, but file isn't being removed. So it can't start again, here is
what I see in /var/log/cloudstack/agent/cloudstack-agent.out file:
2015-05-12 17:04:37,504{GMT} INFO [cloud.agent.AgentShell] (Thread-1:) Agent
started
2015-05-12 17:04:37,506{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
Implementation Version is 4.2.1
2015-05-12 17:04:37,507{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
agent.properties found at /etc/cloudstack/agent/agent.properties
2015-05-12 17:04:37,508{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
Defaulting to using properties file for storage
2015-05-12 17:04:37,509{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
Defaulting to the constant time backoff algorithm
2015-05-12 17:04:37,510{GMT} INFO [cloud.utils.LogUtils] (Thread-1:) log4j
configuration found at /etc/cloudstack/agent/log4j-cloud.xml
2015-05-12 17:04:37,626{GMT} ERROR [cloud.agent.AgentShell] (Thread-1:) Unable
to start agent: Java process is being started twice. If this is not true,
remove /var/run/agent.pid
Unable to start agent: Java process is being started twice. If this is not
true, remove /var/run/agent.pid
I remove this file by hands and do restart again. The agent starts, but
/var/run/agent.pid has zero length again.
This server is just the same as other CentOS hosts (there are many of them in
my farm), but other hosts don't seem to be having this issue.
I opened a ticket (https://issues.apache.org/jira/browse/CLOUDSTACK-8456, there
are more details about the environment), but if anyone has some clues, hints or
ideas, please, share your thoughts on this topic. Any help will be greatly
appreciated! Thanks!
--
V.Melnik
RE: The agent doesn't restart :(
Well, I compared lists of installed packages between this host and other
hosts, and I noticed that I have java-1.8.0 on this server, but other
servers have java-1.7.0. I've downgraded Java and now all is fine. :)
-Original Message-
From: Vladimir Melnik [mailto:v.mel...@uplink.ua]
Sent: Tuesday, May 12, 2015 6:34 PM
To: users@cloudstack.apache.org
Subject: RE: The agent doesn't restart :(
Ah, now I see what's going on. The agent starts, but it doesn't work at all!
2015-05-12 18:12:49,493 INFO [cloud.agent.AgentShell] (Thread-1:null) Agent
started
2015-05-12 18:12:49,495 INFO [cloud.agent.AgentShell] (Thread-1:null)
Implementation Version is 4.2.1
2015-05-12 18:12:49,496 INFO [cloud.agent.AgentShell] (Thread-1:null)
agent.properties found at /etc/cloudstack/agent/agent.properties
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: workers
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: port
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: pod
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: resource
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: private.network.device
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: zone
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: guid
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: guest.network.device
2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: cluster
2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: domr.scripts.dir
2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: local.storage.uuid
2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: public.network.device
2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found
property: host
2015-05-12 18:12:49,499 INFO [cloud.agent.AgentShell] (Thread-1:null)
Defaulting to using properties file for storage
2015-05-12 18:12:49,500 INFO [cloud.agent.AgentShell] (Thread-1:null)
Defaulting to the constant time backoff algorithm
2015-05-12 18:12:49,501 INFO [cloud.utils.LogUtils] (Thread-1:null) log4j
configuration found at /etc/cloudstack/agent/log4j-cloud.xml
2015-05-12 18:12:49,614 DEBUG [cloud.agent.AgentShell] (Thread-1:null)
Checking to see if agent.pid exists.
2015-05-12 18:12:49,622 DEBUG [cloud.utils.ProcessUtil] (Thread-1:null)
Executing: bash -c echo $PPID
And this is the last line I see in the log-file. The file /var/log/agent.pid
is being created, but it has zero length. So, the agent's process is still
working (I can see it by ps), but it doesn't do anything.
Any ideas on the cause of such an odd behavior?
-Original Message-
From: Vladimir Melnik [mailto:v.mel...@uplink.ua]
Sent: Tuesday, May 12, 2015 5:57 PM
To: users@cloudstack.apache.org
Subject: The agent doesn't restart :(
Hello!
I encountered quite an odd problem: the agent doesn't restart on KVM host.
Here is what's going on.
When it starts for the first time it creates /var/run/agent.pid file, but
there are no numbers, the file has zero length.
When I restart it (e.g. by service cloudstack-agent restart command), it's
being shut down, but file isn't being removed. So it can't start again, here
is what I see in /var/log/cloudstack/agent/cloudstack-agent.out file:
2015-05-12 17:04:37,504{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
Agent started
2015-05-12 17:04:37,506{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
Implementation Version is 4.2.1
2015-05-12 17:04:37,507{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
agent.properties found at /etc/cloudstack/agent/agent.properties
2015-05-12 17:04:37,508{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
Defaulting to using properties file for storage
2015-05-12 17:04:37,509{GMT} INFO [cloud.agent.AgentShell] (Thread-1:)
Defaulting to the constant time backoff algorithm
2015-05-12 17:04:37,510{GMT} INFO [cloud.utils.LogUtils] (Thread-1:) log4j
configuration found at /etc/cloudstack/agent/log4j-cloud.xml
2015-05-12 17:04:37,626{GMT} ERROR [cloud.agent.AgentShell] (Thread-1:)
Unable to start agent: Java process is being started twice. If this is not
true, remove /var/run/agent.pid Unable to start agent: Java process is being
started twice. If this is not true, remove /var/run/agent.pid
I remove this file by hands and do restart again. The agent starts, but
/var/run/agent.pid has zero length again.
This server is just the same as other CentOS hosts (there are many of them
in my farm), but other hosts don't seem to be having this issue.
I opened a ticket (https://issues.apache.org/jira/browse/CLOUDSTACK-8456,
there are more details about the environment), but if anyone has some clues,
hints or ideas, please
CephFS vs. GFS2
Dear colleagues, Did anyone compare CephFS and GFS2 or did anyone read any articles about comparison of CephFS and GFS2 in regard to their performance? Alas, I haven't found anything on this topic. Right now I'm using GFS2 as a primary storage filesystem (it's accessible by iSCSI+multipath), but, of course, I'd like to gain more performance. Should I try CephFS as an alternative? Thanks! -- V.Melnik
RE: XenServer
Oh, thank you so much, folks, for your replies on this topic! Now I understand that I shouldn't migrate to XenServer, as I already have a bunch of hosts running KVM and it really won't be a good idea to bread such a zoo. The only thing that makes me feel envy is an ability to change configuration of a virtual machine without restarting it. As I know, KVM doesn't support it yet, am I right? Other XenServer features don't seem to be so exciting to me :) One more time - thanks to all who have helped me to decide not to do something that I would regret. Best regards, V.
MonkeyMan-1.0.0
Dear colleagues, The 1.0.0 version has been released, the development branch has been merged to the stable one. Now you can use bin/makesnapshots.pl to automatically create snapshots by your own schedule. For example, you want snapshots for some volumes to be taken only in the night, but some of them should be taken only on Saturdays, also you want the planner to start not more than 2 parallel jobs for a storagepool, oldest snapshots shall be deleted, but 2 latest snapshots shall be kept, etc... And you can easily configure it, make sure how easy it is: https://github.com/melnik13/monkeyman/blob/stables/etc/backup_schedule.conf.example :) Feel free to download and use MonkeyMan, any feedback will be greatly appreciated. The project's website: http://monkeyman.tucha.ua/ The project's Twitter: https://twitter.com/monkey13man -- V.Melnik
MonkeyMan - a perl5 library for managing your Apache CloudStack installation
Dear colleagues,
If you administrating CloudStack installations, you may find it useful.
If you love Perl5, you may consider it interesting.
I'm developing a library and some set of tools for managing
CloudStack-based infrastructure from the command line. It's going to
become a smart system for doing lots of administrative tasks, so it's
only the beginning for now. :-)
But what we have at the moment?
As about tools, we can do such things from UNIX-shell...
For example, let's assume you want to check state of the VM having
a certain IP-address AND being a member of a certain domain:
admin mm_vm_info has_ipaddress=10.1.1.127 has_domain=A201306
?xml version=1.0 encoding=UTF-8?
listvirtualmachinesresponse
virtualmachine
id99b885d4-70d7-4efc-8a4e-53417893fb19/id
name99b885d4-70d7-4efc-8a4e-53417893fb19/name
[...]
Or, if you want to, you can get certain parameters of this output:
admin mm_vm_info has_ipaddress=10.1.1.127 has_domain=A201306 \
-x //state -x //hostname -x //instancename
stateRunning/state
hostnameh2.c1.p1.z1.tucha13.net/hostname
instancenamei-51-135-VM/instancename
You can reset the VM found by your desired criterias:
admin mm_vm_reset has_instancename=i-13-666-VM
Want to listen to someone's network interface? No problem.
admin mm_vm_tcpdump has_ipaddress=10.1.1.253
tcpdump: WARNING: vnet17: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vnet17, link-type EN10MB (Ethernet), capture size 65535 bytes
15:21:01.845886 IP 10.1.1.1.36686 10.1.1.253.imaps: Flags [S], seq 316746440,
win 14600, options [mss 1460,sackOK,TS val 1959872931 ecr 0,nop,wscale 6],
length 0
15:21:01.846023 IP 10.1.1.253.imaps 10.1.1.1.36686: Flags [S.], seq
2234071684, ack 316746441, win 14480, options [mss 1460,sackOK,TS val
2579908884 ecr 1959872931,nop,wscale 6], length 0
^^^ The script have found the hostname and the instance name, logged
into the host, analyzed output of virsh dumpxml ..., found the
interface's name (vnet17) and launched tcpdump.
And you can use all these things not only from the command-line, as you
can use the object oriented library using all these things from your own
Perl5 scripts.
Let's assume you want to find and load all the information about some
domain:
my $domain = eval { MonkeyMan::CloudStack::Elements::Domain-new(
mm = $mm,
load_dom= {
conditions = {
path= 'ROOT/CUSTOMERS/ZALOOPA'
}
}
)};
if($@) { $log-warn(Can't
MonkeyMan::CloudStack::Elements::Domain-new(): $@); next; }
Voila, now you have the corresponding object's reference in the $domain
variable. You can do some easy tricks with that domain. It's pretty easy
to get any parameter:
my $domain_id = $domain-get_parameter('id');
unless(defined($domain_id)) {
$log-warn(Can't get the ID of the domain .
($domain-has_error ? (: . $domain-error_message) :
undef)
);
next;
}
What if you want to get all volumes belongs to this domain? It's easy:
my $volumz = $domain-find_related_to_me(volume);
$log-logdie($domain-error_message) unless defined($volumz);
No kidding, you have the reference to the list of XML::LibXML documents
who have the domainid parameter corresponding to this domain. You can
easily initialize them as objects to do other cool things with these
volumes:
foreach my $volume_dom (@{ $volumz }) {
my $volume = eval {
MonkeyMan::CloudStack::Elements::Volume-new(
mm = $mm,
load_dom= {
dom= $volume_dom # the XML document
}
); };
if($@) { $log-warn(Can't
MonkeyMan::CloudStack::Elements::Volume-new(): $@); next; }
Oh, well, too much words... :-)
Would you like to use it? You're strongly welcome:
https://github.com/melnik13/monkeyman/
Would you like to develop it with me? Feel free to drop me a line.
A great week to you all!
--
V.Melnik
Re: MonkeyMan - a perl5 library for managing your Apache CloudStack installation
On Mon, Jun 30, 2014 at 02:59:12PM +0200, sebgoa wrote: Would you like to develop it with me? Feel free to drop me a line. you might want to forward this mail to the dev list. I'll do! A great week to you all! Awesome work, thanks for sharing Thank you! Always glad to share :) -- V.Melnik
Re: MonkeyMan - a perl5 library for managing your Apache CloudStack installation
Oh, I like it too, thanks to Danil Kolesnichenko d.kolesniche...@tucha.ua for the artwork! On Mon, Jun 30, 2014 at 10:17:34PM +0530, Rohit Yadav wrote: Thanks for sharing Vladimir, love the mascot :) Cheers. -- V.Melnik
numCpusGood: false
Dear colleagues, Is there any way to bypass numCpusGood checking? Thank you! V.
Multiple secondary storages in a zone
Dear colleagues, How ACS-4.x decides which secondary storage to use when making a snapshot? Is there any hacks to use without making changes in sources? I'd like to store snapshots and templates on different secondary storages, is it possible? Thanks in advance for any hints, ideas and suggestions! V.
RE: Snapshots on KVM host
THANK. YOU. SO. MUCH! :) Too bad that there were any mentions in log-files about that attempt. -Original Message- From: Nux! [mailto:n...@li.nux.ro] Sent: Thursday, February 27, 2014 3:04 PM To: users@cloudstack.apache.org Subject: Re: Snapshots on KVM host On 27.02.2014 12:59, Vladimir Melnik wrote: Dear colleagues, Today is the 3rd day I'm fighting this issue, so I any hints will be greatly appreciated. :) Hello Vladimir, Check this http://www.nux.ro/archive/2014/01/Taking_KVM_volume_snapshots_with_Cloudstack_4_2_on_CentOS_6_5.html HTH Lucian -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
RE: Snapshots on KVM host
Oh, thank you! -Original Message- From: Nux! [mailto:n...@li.nux.ro] Sent: Thursday, February 27, 2014 3:45 PM To: users@cloudstack.apache.org Subject: RE: Snapshots on KVM host On 27.02.2014 13:16, Vladimir Melnik wrote: THANK. YOU. SO. MUCH! :) Too bad that there were any mentions in log-files about that attempt. You have to increase the verbosity of your log files. ie sed -i 's/INFO/DEBUG/g' /etc/cloudstack/agent/log4j-cloud.xml -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
RE: httpd on console proxy VM
Oops! :) Thank you so much. So, who should be listening on the 443? -Original Message- From: Erik Weber [mailto:terbol...@gmail.com] Sent: Tuesday, February 25, 2014 2:17 PM To: users@cloudstack.apache.org Subject: Re: httpd on console proxy VM I have the same, but I don't think apache is being used. atleast i don't have any apache2-processes running on my cpvm, while I do see java bind to port 443 -- Erik On Tue, Feb 25, 2014 at 1:13 PM, Vladimir Melnik v.mel...@tucha.co.ukwrote: I have to stress, that I don't even have 10\.1\.1 in the dump of my CloudStack's database, so I do NOT use it anywhere in my configuration (yes, I use another prefix as the guest's CIDR). But httpd is trying to bind to 10.1.1.1:80 even after destroying this VM. :( -Original Message- From: Vladimir Melnik [mailto:v.mel...@tucha.co.uk] Sent: Tuesday, February 25, 2014 2:02 PM To: users@cloudstack.apache.org Subject: httpd on console proxy VM Dear colleagues, I'm installing CloudStack-4.2.1 on the new setup and I've faced an odd thing. Alas, httpd on the console proxy VM doesn't start, because it's trying to bind to 10.1.1.1 address, but the instance does NOT have such address bound to any of its' interfaces, and that's okay, because I have such configuration: lo: 127.0.0.1/8 eh0: 169.254.1.72/16 eth1: 172.24.64.250/24 eth2: x.x.x.x/24 But in httpd's configuration I can see lots of mentions of 10.1.1.1: /etc/apache2/ports.conf:NameVirtualHost 10.1.1.1:80 /etc/apache2/ports.conf:Listen 10.1.1.1:80 /etc/apache2/ports.conf:Listen 10.1.1.1:443 /etc/apache2/ports.conf:Listen 10.1.1.1:443 /etc/apache2/sites-available/default.orig:VirtualHost 10.1.1.1:80 /etc/apache2/sites-available/default-ssl.orig:VirtualHost 10.1.1.1:443 /etc/apache2/sites-available/default:VirtualHost 10.1.1.1:80 /etc/apache2/sites-available/default-ssl:VirtualHost 10.1.1.1:443 /etc/apache2/vhostexample.conf:VirtualHost 10.1.1.1:80 /etc/apache2/vhostexample.conf:VirtualHost 10.1.1.1:443 /etc/apache2/vhostexample.conf:Listen 10.1.1.1:80 /etc/apache2/vhostexample.conf:Listen 10.1.1.1:443 /etc/apache2/vhostexample.conf:Listen 10.1.1.1:443 Where does it take 10.1.1.1? As I understand, CloudStack should have change it while deploying this VM, but it hasn't. :( Of course, I tried to delete and create console proxy VM, but it doesn't help. Where is the glitch? Thank you! -- With best regards, wishes and so on, Vladimir Melnik :)
RE: httpd on console proxy VM
Thank you! I've found out where was the problem: the agent didn't start, because it wasn't able to connect the management server on 8290/tcp, but I fixed it and everything's fine now. Have a nice day! -Original Message- From: Erdősi Péter [mailto:f...@niif.hu] Sent: Tuesday, February 25, 2014 2:39 PM To: users@cloudstack.apache.org Subject: Re: httpd on console proxy VM Hy! 2014.02.25. 13:30 keltezéssel, Vladimir Melnik írta: Oops! :) Thank you so much. So, who should be listening on the 443? My proxys vm said: netstat -alnp |grep 443 tcp0 0 0.0.0.0:443 0.0.0.0:* LISTEN 3758/java Dear: Fazy
sdX-devices became vdX-devices in CloudStack-4.2.0
Dear colleagues, I'm planning to migrate to CS-4.2.0, but I noticed one pitfall. When I tried to copy one VM to the new environment from CS-2.2.14, the operating system inside the VM didn't boot properly, because all sda-devices became vda-devices. Are there any hacks to prevent it? Thank you!
Backing up volumes on KVM hypervisors
Dear colleagues, As far as I understand, CloudStack doesn't support volume snapshots in KVM clusters anymore. Does anyone use any dirty hacks for backing up volumes? -- V.Melnik
RE: Backing up volumes on KVM hypervisors
Thank you so much! I didn't know it has been disabled. -Original Message- From: Nux! [mailto:n...@li.nux.ro] Sent: Monday, November 18, 2013 2:41 PM To: users@cloudstack.apache.org Subject: Re: Backing up volumes on KVM hypervisors On 18.11.2013 11:27, Vladimir Melnik wrote: Dear colleagues, As far as I understand, CloudStack doesn't support volume snapshots in KVM clusters anymore. Does anyone use any dirty hacks for backing up volumes? Hi, That's not correct, you can snapshot (back up) root and data disks from the Storage UI, you just can't snapshot the VM as a whole. Go in Global Settings and enable KVM snapshots (no idea why this is disabled by default).. -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro
Starting system vms
Dear colleagues, I installed CloudStack-4.2.0 in a lab. It has been working for a while (it even started system vms), but after the last reboot it doesn't start system vms: 2013-11-14 14:32:11,013 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) System vm template is not ready at data center 2, wait until it is ready to launch secondary storage vm 2013-11-14 14:32:11,013 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) Zone 2 is not ready to launch secondary storage VM yet 2013-11-14 14:32:11,196 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) System vm template is not ready at data center 2, wait until it is ready to launch console proxy vm 2013-11-144:32:11,196 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) Zone 2 is not ready to launch console proxy yet As I can see, the template is present, the secondary storage is available (I can mount it by NFS from both of hypervisors) and I don't see any errors neither in cloudstack-management.log nor in agent.log nor in libvirtd.log. Kindly grateful for any suggestions. Thank you! -- V.Melnik
RE: Starting system vms
Had to delete the secondary storage from the configuration and reconfigure it. -Original Message- From: Sanjeev Neelarapu [mailto:sanjeev.neelar...@citrix.com] Sent: Thursday, November 14, 2013 2:46 PM To: users@cloudstack.apache.org Subject: RE: Starting system vms Any mess-up happened with cloud DB? Please check the template status in template_store_ref table, should be in DOWNLOADED state. -Original Message- From: Vladimir Melnik [mailto:v.mel...@uplink.ua] Sent: Thursday, November 14, 2013 6:10 PM To: users@cloudstack.apache.org Subject: Starting system vms Dear colleagues, I installed CloudStack-4.2.0 in a lab. It has been working for a while (it even started system vms), but after the last reboot it doesn't start system vms: 2013-11-14 14:32:11,013 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) System vm template is not ready at data center 2, wait until it is ready to launch secondary storage vm 2013-11-14 14:32:11,013 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) Zone 2 is not ready to launch secondary storage VM yet 2013-11-14 14:32:11,196 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) System vm template is not ready at data center 2, wait until it is ready to launch console proxy vm 2013-11-144:32:11,196 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) Zone 2 is not ready to launch console proxy yet As I can see, the template is present, the secondary storage is available (I can mount it by NFS from both of hypervisors) and I don't see any errors neither in cloudstack-management.log nor in agent.log nor in libvirtd.log. Kindly grateful for any suggestions. Thank you! -- V.Melnik
public, private and guest interfaces
Dear colleagues, It seems, I need some help again. How to understand what interface should be used as public, private or guest interface? On each hypervisor I have one bonded interface (bond0). Also I have eth0.101 with 192.168.101.xxx/24 designated for management (192.168.101.0/24 is designated to be the internal network of the pod). When system VMs start they have 192.168.101.11 and 192.168.101.13 addresses as their Private IP addresses. So, did I get this right that bond0.101 should be bridged by some cloudbrX and this cloudbrX should be stated as private.network.device, right? What interface should be stated as guest.network.device? Some cloudbrX which will be bridged with the whole bond0, right? And what interface should be stated as public.network.device? Thank you very much! -- V.Melnik
RE: HA for Cloudstack
...also you can use SharedMountPoint on KVM hypervisors and mount GFS2-filesystem connected via iSCSI. It requires some time to understand how to manage with redhat-cluster, but it's definitely worth do implement! :-) -- V.Melnik -Original Message- From: Chip Childers [mailto:chipchild...@apache.org] Sent: Thursday, November 14, 2013 5:12 PM To: users@cloudstack.apache.org; jk744 Subject: Re: HA for Cloudstack On Thu, Nov 14, 2013 at 09:18:40AM +0800, jk744 wrote: Dear users, Could you please tell me where to get HA setup manual for Cloudstack VM ? Is it available for free version Cloudstack? Thanks a lot. Best regards. Jack The documentation for installation can be found here: http://cloudstack.apache.org/docs/en-US/index.html That should explain a multi-management server setup for you.
RE: HA for Cloudstack
Sorry, it supposed to be a reply for Getting Help topic. :) -- V.Melnik -Original Message- From: Vladimir Melnik [mailto:v.mel...@uplink.ua] Sent: Thursday, November 14, 2013 5:16 PM To: users@cloudstack.apache.org Subject: RE: HA for Cloudstack ...also you can use SharedMountPoint on KVM hypervisors and mount GFS2-filesystem connected via iSCSI. It requires some time to understand how to manage with redhat-cluster, but it's definitely worth do implement! :-) -- V.Melnik -Original Message- From: Chip Childers [mailto:chipchild...@apache.org] Sent: Thursday, November 14, 2013 5:12 PM To: users@cloudstack.apache.org; jk744 Subject: Re: HA for Cloudstack On Thu, Nov 14, 2013 at 09:18:40AM +0800, jk744 wrote: Dear users, Could you please tell me where to get HA setup manual for Cloudstack VM ? Is it available for free version Cloudstack? Thanks a lot. Best regards. Jack The documentation for installation can be found here: http://cloudstack.apache.org/docs/en-US/index.html That should explain a multi-management server setup for you.
Using XenServer-6.2.0 and CloudStack-4.2.0
Hi there, Does anybody use XenServer-6.2.0 from xenserver.org? Does it work with CloudStack-4.2.0? Did you have any troubles with that? Thank you! -- V.Melnik
Can't compile CloudStack-4.2:
Dear colleagues, I'm using 2.2.14 and it holds about a hundred of instances). I decided to try 4.2.0 in the testing lab, but I encountered a problem while I was trying to build RPMs. That's what I've got: [INFO] Apache CloudStack Console Proxy ... SUCCESS [37.729s] [INFO] Apache CloudStack Client UI ... SUCCESS [30.354s] [INFO] Apache CloudStack Console Proxy Plugin SUCCESS [0.840s] [INFO] Apache CloudStack AWS API Bridge .. FAILURE [49.932s] [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 12:16.096s [INFO] Finished at: Sat Oct 26 14:13:44 EEST 2013 [INFO] Final Memory: 55M/246M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project cloud-awsapi: Compilation failure: Compilation failure: [ERROR] error: error reading /root/.m2/repository/org/apache/axis2/mex/1.5.4/mex-1.5.4-impl.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/org/apache/axis2/axis2-mtompolicy/1.5.4/axis2-mtompolic y-1.5.4.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/org/apache/ws/commons/axiom/axiom-dom/1.2.10/axiom-dom- 1.2.10.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/org/opensaml/opensaml1/1.1/opensaml1-1.1.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/commons-lang/commons-lang/2.3/commons-lang-2.3.jar; error in opening zip file As I can see the problem is in downloading these JAR-files: Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/axis2/mex/1.5.4/ mex-1.5.4-impl.jar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/axis2/axis2-mtom policy/1.5.4/axis2-mtompolicy-1.5.4.jar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/axis2/addressing /1.5.4/addressing-1.5.4.mar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/ws/commons/axiom /axiom-dom/1.2.10/axiom-dom-1.2.10.jar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/opensaml/opensaml1/1.1/ opensaml1-1.1.jar I have several [WARNING]s (the builder said that Checksum validation failed) exactly for these files and, as I can see, these files aren't there anymore. Honestly, Java is such a dark forest for me, so, please, can anyone advice how to fix these links? Thank you very much! -- V.Melnik
RE: Can't compile CloudStack-4.2:
Thank you, I'll try to download these JAR-files manually! Why didn't I tried it? I thought, mvn will try to re-download them and fail again. -Original Message- From: Bjoern Teipel [mailto:bjoern.tei...@internetbrands.com] Sent: Tuesday, October 29, 2013 6:31 PM To: users@cloudstack.apache.org Subject: Re: Can't compile CloudStack-4.2: I had issues with similar jars and here my quick fix: cd $HOME/.m2/repository/org/apache/axis2/axis2-mtompolicy/1.5.4/ \ wget http://repo1.maven.org/maven2/org/apache/axis2/axis2-mtompolicy/1.5.4/axis2- mtompolicy-1.5.4.jar -O axis2-mtompolicy-1.5.4.jar cd $HOME/.m2/repository/org/apache/ws/commons/axiom/axiom-dom/1.2.10 \ wget http://repo1.maven.org/maven2/org/apache/ws/commons/axiom/axiom-dom/1.2.10/a xiom-dom-1.2.10.jar -O axiom-dom-1.2.10.jar cd $HOME/.m2/repository/org/opensaml/opensaml1/1.1 \ wget http://repo1.maven.org/maven2/org/opensaml/opensaml/1.1/opensaml-1.1.jar -O opensaml1-1.1.jar cd $HOME/.m2/repository/commons-lang/commons-lang/2.3 \ wget http://repo1.maven.org/maven2/commons-lang/commons-lang/2.3/commons-lang-2.3 .jar -O commons-lang-2.3.jar cd $HOME/.m2/repository/bouncycastle/bcprov-jdk14/140 \ wget http://repo1.maven.org/maven2/bouncycastle/bcprov-jdk14/140/bcprov-jdk14-140 .jar -O bcprov-jdk14-140.jar On 10/29/2013 02:10 AM, Vladimir Melnik wrote: Dear colleagues, I'm using 2.2.14 and it holds about a hundred of instances). I decided to try 4.2.0 in the testing lab, but I encountered a problem while I was trying to build RPMs. That's what I've got: [INFO] Apache CloudStack Console Proxy ... SUCCESS [37.729s] [INFO] Apache CloudStack Client UI ... SUCCESS [30.354s] [INFO] Apache CloudStack Console Proxy Plugin SUCCESS [0.840s] [INFO] Apache CloudStack AWS API Bridge .. FAILURE [49.932s] [INFO] -- -- [INFO] BUILD FAILURE [INFO] -- -- [INFO] Total time: 12:16.096s [INFO] Finished at: Sat Oct 26 14:13:44 EEST 2013 [INFO] Final Memory: 55M/246M [INFO] -- -- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project cloud-awsapi: Compilation failure: Compilation failure: [ERROR] error: error reading /root/.m2/repository/org/apache/axis2/mex/1.5.4/mex-1.5.4-impl.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/org/apache/axis2/axis2-mtompolicy/1.5.4/axis2-mto mpolic y-1.5.4.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/org/apache/ws/commons/axiom/axiom-dom/1.2.10/axio m-dom- 1.2.10.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/org/opensaml/opensaml1/1.1/opensaml1-1.1.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/commons-lang/commons-lang/2.3/commons-lang-2.3.ja r; error in opening zip file As I can see the problem is in downloading these JAR-files: Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/axis2/mex/ 1.5.4/ mex-1.5.4-impl.jar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/axis2/axis 2-mtom policy/1.5.4/axis2-mtompolicy-1.5.4.jar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/axis2/addr essing /1.5.4/addressing-1.5.4.mar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/ws/commons /axiom /axiom-dom/1.2.10/axiom-dom-1.2.10.jar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/opensaml/opensaml 1/1.1/ opensaml1-1.1.jar I have several [WARNING]s (the builder said that Checksum validation failed) exactly for these files and, as I can see, these files aren't there anymore. Honestly, Java is such a dark forest for me, so, please, can anyone advice how to fix these links? Thank you very much! -- Bjoern Teipel System Operation Manager UNIX Internet Brands 909 N. Sepulveda Blvd., 11th Floor El Segundo, CA 90245 P: (310) 280- 5287 bjoern.tei...@internetbrands.com
qemu-kvm tweaking (CloudStack-2.2.14)
Dear colleagues, Has anybody managed to set/change parameters of qemu-kvm for certain instance before it launches? For example, if I want to add the 3rd drive or to emulate some different model of the NIC and so on... Are there any working hacks for that? Thanks! -- V.Melnik
