[GitHub] cloudstack issue #977: [4.9] CLOUDSTACK-8746: VM Snapshotting implementation...

[ustcweizhou]

Github user ustcweizhou commented on the issue:

https://github.com/apache/cloudstack/pull/977
  
rebased with latest master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1843: WIP: instrumeted code

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1843
  
Packaging result: ✔centos6 ✔centos7 ✔debian. JID-405


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: patchviasocket seems to be broken with qemu 2.3(+?)

[Syahrul Sazli Shaharir]

On 2016-12-20 17:53, Wei ZHOU wrote:

Hi Synhrul,

Could you upload the /var/log/cloud.log ?


Sure:-

Working router VM: http://pastebin.com/hwwk86ve

Non-working router VM: http://pastebin.com/G4nv09ab

Thanks.



-Wei

2016-12-20 3:18 GMT+01:00 Syahrul Sazli Shaharir :


On 2016-12-19 18:10, Syahrul Sazli Shaharir wrote:


On 2016-12-19 17:03, Linas Žilinskas wrote:

From the logs it doesn't seem that the script timeouts. "Execution 
is

successful", so it manages to pass the data over the socket.

I guess the systemvm just doesn't configure itself for some reason.



You are right, I was able to enter the router VM console at some 
point
during the timeout loops, and able to capture syslog output during 
the

loop:-

http://pastebin.com/n37aHeSa



I restarted another network, and that network's router VM was able to 
be
recreated, even on the same host as the failed network (and both 
networks

are exactly same configuration, only VLAN & subnet are different).
Comparing between the two syslog outputs during boot shows the 
problematic

network router VM self-configuration got stuck in vm_dhcp_entry.json .

1. Working network router VM : http://pastebin.com/Y6zpDa6M
2. Non-working network router VM : http://pastebin.com/jzfGMGQB

Thanks.




Also, in my personal tests, I noticed some different behaviour with
different kernels. Don't remember the specifics right now, but on 
some
combinations (qemu / kernel) the socket acted differently. For 
example

the data was sent over the socket, but wasn't visible inside the VM.
Other times the socket would be stuck from the host side.

So i would suggest testing different kernels (3.x, 4.4.x, 4.8.x) or
try to login to the system vm and see what's happening from inside.



Will do this next and feedback the results here.

Thanks for your help! :)


On 12/16/16 03:46, Syahrul Sazli Shaharir wrote:


On 2016-12-16 11:27, Syahrul Sazli Shaharir wrote:

On Wed, 26 Oct 2016, Linas ?ilinskas wrote:

So after some investigation I've found out that qemu 2.3.0 is 
indeed

broken, at least the way CS uses the qemu chardev/socket.

Not sure in which specific version it happened, but it was fixed in
2.4.0-rc3, specifically noting that CloudStack 4.2 was not working.

qemu git commit: 4bf1cb03fbc43b0055af60d4ff093d6894aa4338

Also attaching the patch from that commit.

For our own purposes i've included the patch to the qemu-kvm-ev
package (2.3.0) and all is well.

Hi,

I am facing the exact same issue on latest Cloudstack 4.9.0.1, on
latest CentOS 7.3.1611, with latest qemu-kvm-ev-2.6.0-27.1.el7
package.

The issue initially surfaced following a heartbeat-induced reset of
all hosts, when it was on CS 4.8 @ CentOS 7.0 and stock
qemu-kvm-1.5.3. Since then, the patchviasocket.pl/py timeouts
persisted for 1 out of 4 router VM/networks, even after upgrading 
to


latest code. (I have checked the qemu-kvm-ev-2.6.0-27.1.el7 source,
and the patched code are pretty much still intact, as per the
2.4.0-rc3 commit).

Any help would be greatly appreciated.

Thanks.

(Attached are some debug logs from the host's agent.log)



Here are the debug logs as mentioned: http://pastebin.com/yHdsMNzZ

Thanks.

--sazli


On 2016-10-20 09:59, Linas ?ilinskas wrote:

Hi.

We have made an upgrade to 4.9.

Custom build packages with our own patches, which in my mind (i'm
the only
one patching those) should not affect the issue i'll describe.

I'm not sure whether we didn't notice it before, or it's actually
related
to something in 4.9

Basically our system vm's were unable to be patched via the qemu
socket.
The script simply error'ed out with a timeout while trying to push
the
data to the socket.

Executing it manually (with cmd line from the logs) resulted the
same. I
even tried the old perl variant, which also had same result.

So finally we found out that this issue happens only on our HVs
which run
qemu 2.3.0, from the centos 7 special interest virtualization repo.
Other
ones that run qemu 1.5, from official repos, can patch the system
vms
fine.

So i'm wondering if anyone tested 4.9 with kvm with qemu >= 2.x?
Maybe it
something else special in our setup. e.g. we're running the HVs
from a
preconfigured netboot image (pxe), but all of them, including those
with
qemu 1.5, so i have no idea.

Linas ?ilinskas
Head of Development
website  [1] facebook
 [2] twitter
 [3] linkedin

[4]

Host1Plus is a division of Digital Energy Technologies Ltd.

26 York Street, London W1U 6PZ, United Kingdom



--
--sazli

[GitHub] cloudstack issue #1843: WIP: instrumeted code

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1843
  
@abhinandanprateek a Jenkins job has been kicked to build packages. I'll 
keep you posted as I make progress.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1843: WIP: instrumeted code

[abhinandanprateek]

Github user abhinandanprateek commented on the issue:

https://github.com/apache/cloudstack/pull/1843
  
@blueorangutan package


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1843: WIP: instrumeted code

[abhinandanprateek]

GitHub user abhinandanprateek opened a pull request:

https://github.com/apache/cloudstack/pull/1843

WIP: instrumeted code

WIP

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shapeblue/cloudstack CLOUDSTACK-9687

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1843.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1843


commit 861944c78246cc8a9a9ecafeb4f88c6b6d48f3e0
Author: Abhinandan Prateek 
Date:   2016-12-21T06:09:06Z

WIP: instrumeted code




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1842: CLOUDSTACK-9686: Fixed multiple entires for b...

[anshul1886]

GitHub user anshul1886 opened a pull request:

https://github.com/apache/cloudstack/pull/1842

CLOUDSTACK-9686: Fixed multiple entires for builtin template in template

store ref table so builtin template is never downloaded completely
 In handleSysTemplateDownload method creating template only if there exists 
no entry
handleTemplateSync will take care of other scenario

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/anshul1886/cloudstack-1 CLOUDSTACK-9686

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1842.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1842


commit 929595c114f1214f064419a305cc115a3e136803
Author: Anshul Gangwar 
Date:   2016-01-08T07:58:11Z

CLOUDSTACK-9686: Fixed multiple entires for builtin template in template
store ref table so builtin template is never downloaded completely
 In handleSysTemplateDownload method creating template only if there exists 
no entry
handleTemplateSync will take care of other scenario




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1837: [4.9] Smoketest Health

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1837
  
@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos6) has been 
kicked to run smoke tests


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1837: [4.9] Smoketest Health

[rhtyd]

Github user rhtyd commented on the issue:

https://github.com/apache/cloudstack/pull/1837
  
@blueorangutan test centos7 kvm-centos6


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

RE: Current status of IPv6 in Basic Networking

[Simon Weller]

Agreed. Great work!

Simon Weller/ENA
(615) 312-6068

-Original Message-
From: Pierre-Luc Dion [pdion...@apache.org]
Received: Tuesday, 20 Dec 2016, 4:42PM
To: dev@cloudstack.apache.org [dev@cloudstack.apache.org]
Subject: Re: Current status of IPv6 in Basic Networking

Thanks Wido,

It's cool seeing update like this!

On Dec 20, 2016 15:48, "Wido den Hollander"  wrote:

> Hi,
>
> Just wanted to give an update on my IPv6 progress since my last [0] update.
>
> The Wiki [1] still contains the most data about this, but the first PR [2]
> is already open which should fix CLOUDSTACK-9359 [3].
>
> When PR #1700 merges (I hope in to 4.10!) I will submit a second PR which
> brings full Security Group support for IPv6 under KVM with Basic Networking.
>
> The code [4] is already running on a private cloud at PCextreme and
> working fine there. I really hope that code can also make it into 4.10
> since that will be a huge IPv6 step for CloudStack.
>
> Afterwards I will be working on getting better IPv6 support for the SSVMs.
> Main focus is still Basic Networking however. Anything additional is a good
> side-effect.
>
> I ask everybody to look at PR #1700 and give feedback. The sooner it
> merges, the better.
>
> Thanks!
>
> Wido
>
> [0]: http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
> 201610.mbox/%3C803597888.5548.1475520180772%40ox.pcextreme.nl%3E
> [1]: https://cwiki.apache.org/confluence/display/CLOUDSTACK/
> IPv6+in+Basic+Networking
> [2]: https://github.com/apache/cloudstack/pull/1700
> [3]: https://issues.apache.org/jira/browse/CLOUDSTACK-9359
> [4]: https://github.com/wido/cloudstack/commits/ipv6-basic-
> networking-secgroup-ports
>

Re: Current status of IPv6 in Basic Networking

[Pierre-Luc Dion]

Thanks Wido,

It's cool seeing update like this!

On Dec 20, 2016 15:48, "Wido den Hollander"  wrote:

> Hi,
>
> Just wanted to give an update on my IPv6 progress since my last [0] update.
>
> The Wiki [1] still contains the most data about this, but the first PR [2]
> is already open which should fix CLOUDSTACK-9359 [3].
>
> When PR #1700 merges (I hope in to 4.10!) I will submit a second PR which
> brings full Security Group support for IPv6 under KVM with Basic Networking.
>
> The code [4] is already running on a private cloud at PCextreme and
> working fine there. I really hope that code can also make it into 4.10
> since that will be a huge IPv6 step for CloudStack.
>
> Afterwards I will be working on getting better IPv6 support for the SSVMs.
> Main focus is still Basic Networking however. Anything additional is a good
> side-effect.
>
> I ask everybody to look at PR #1700 and give feedback. The sooner it
> merges, the better.
>
> Thanks!
>
> Wido
>
> [0]: http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
> 201610.mbox/%3C803597888.5548.1475520180772%40ox.pcextreme.nl%3E
> [1]: https://cwiki.apache.org/confluence/display/CLOUDSTACK/
> IPv6+in+Basic+Networking
> [2]: https://github.com/apache/cloudstack/pull/1700
> [3]: https://issues.apache.org/jira/browse/CLOUDSTACK-9359
> [4]: https://github.com/wido/cloudstack/commits/ipv6-basic-
> networking-secgroup-ports
>

Current status of IPv6 in Basic Networking

[Wido den Hollander]

Hi,

Just wanted to give an update on my IPv6 progress since my last [0] update.

The Wiki [1] still contains the most data about this, but the first PR [2] is 
already open which should fix CLOUDSTACK-9359 [3].

When PR #1700 merges (I hope in to 4.10!) I will submit a second PR which 
brings full Security Group support for IPv6 under KVM with Basic Networking.

The code [4] is already running on a private cloud at PCextreme and working 
fine there. I really hope that code can also make it into 4.10 since that will 
be a huge IPv6 step for CloudStack.

Afterwards I will be working on getting better IPv6 support for the SSVMs. Main 
focus is still Basic Networking however. Anything additional is a good 
side-effect.

I ask everybody to look at PR #1700 and give feedback. The sooner it merges, 
the better.

Thanks!

Wido

[0]: 
http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201610.mbox/%3C803597888.5548.1475520180772%40ox.pcextreme.nl%3E
[1]: 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/IPv6+in+Basic+Networking
[2]: https://github.com/apache/cloudstack/pull/1700
[3]: https://issues.apache.org/jira/browse/CLOUDSTACK-9359
[4]: 
https://github.com/wido/cloudstack/commits/ipv6-basic-networking-secgroup-ports

[GitHub] cloudstack pull request #1841: CLOUDSTACK-9684 Invalid zone id error while l...

[sateesh-chodapuneedi]

GitHub user sateesh-chodapuneedi opened a pull request:

https://github.com/apache/cloudstack/pull/1841

CLOUDSTACK-9684 Invalid zone id error while listing vmware zone

Issue
=
While listing datacenters associated with a zone, only zone Id validation 
is required.
There is no need to have additional checks like zone is a legacy zone or 
not.

Fix
===
Removed unnecessary checks over zone ID and just checking if zone with 
specified ID exists or not.

Signed-off-by: Sateesh Chodapuneedi 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sateesh-chodapuneedi/cloudstack 
pr-cloudstack-9684

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1841.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1841


commit 870c2f59b476276475d2c8c711f2aa116ab70a45
Author: Sateesh Chodapuneedi 
Date:   2016-12-20T02:15:26Z

CLOUDSTACK-9684 Invalid zone id error while listing vmware zone
Issue
=
While listing datacenters associated with a zone, only zone Id validation 
is required.
There is no need to have additional checks like zone is a legacy zone or 
not.

Fix
===
Removed unnecessary checks over zone ID and just checking if zone with 
specified ID exists or not.

Signed-off-by: Sateesh Chodapuneedi 




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1837: [4.9] Smoketest Health

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1837
  
Trillian test result (tid-692)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 29989 seconds
Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr1837-t692-kvm-centos7.zip
Test completed. 43 look ok, 6 have error(s)


Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_02_redundant_VPC_default_routes | `Failure` | 862.70 | 
test_vpc_redundant.py
test_04_rvpc_privategw_static_routes | `Failure` | 354.24 | 
test_privategw_acl.py
test_09_delete_detached_volume | `Error` | 10.15 | test_volumes.py
test_08_resize_volume | `Error` | 5.07 | test_volumes.py
test_07_resize_fail | `Error` | 10.19 | test_volumes.py
test_06_download_detached_volume | `Error` | 5.07 | test_volumes.py
test_05_detach_volume | `Error` | 5.07 | test_volumes.py
test_04_delete_attached_volume | `Error` | 5.06 | test_volumes.py
test_03_download_attached_volume | `Error` | 5.08 | test_volumes.py
test_01_create_volume | `Error` | 279.45 | test_volumes.py
ContextSuite context=TestTemplates>:setup | `Error` | 323.20 | 
test_templates.py
ContextSuite context=TestListIdsParams>:setup | `Error` | 0.00 | 
test_list_ids_parameter.py
test_01_vpc_site2site_vpn | Success | 154.40 | test_vpc_vpn.py
test_01_vpc_remote_access_vpn | Success | 65.84 | test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpn | Success | 266.34 | test_vpc_vpn.py
test_02_VPC_default_routes | Success | 461.74 | test_vpc_router_nics.py
test_01_VPC_nics_after_destroy | Success | 873.96 | test_vpc_router_nics.py
test_05_rvpc_multi_tiers | Success | 505.35 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | Success | 1442.99 | 
test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | 
Success | 552.30 | test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | Success | 1277.80 | 
test_vpc_redundant.py
test_02_attach_volume | Success | 50.95 | test_volumes.py
test_deploy_vm_multiple | Success | 357.35 | test_vm_life_cycle.py
test_deploy_vm | Success | 0.02 | test_vm_life_cycle.py
test_advZoneVirtualRouter | Success | 0.02 | test_vm_life_cycle.py
test_10_attachAndDetach_iso | Success | 26.52 | test_vm_life_cycle.py
test_09_expunge_vm | Success | 125.19 | test_vm_life_cycle.py
test_08_migrate_vm | Success | 40.66 | test_vm_life_cycle.py
test_07_restore_vm | Success | 0.08 | test_vm_life_cycle.py
test_06_destroy_vm | Success | 130.65 | test_vm_life_cycle.py
test_03_reboot_vm | Success | 125.65 | test_vm_life_cycle.py
test_02_start_vm | Success | 10.15 | test_vm_life_cycle.py
test_01_stop_vm | Success | 125.63 | test_vm_life_cycle.py
test_CreateTemplateWithDuplicateName | Success | 70.48 | test_templates.py
test_01_create_template | Success | 55.40 | test_templates.py
test_10_destroy_cpvm | Success | 162.36 | test_ssvm.py
test_09_destroy_ssvm | Success | 163.52 | test_ssvm.py
test_08_reboot_cpvm | Success | 131.51 | test_ssvm.py
test_07_reboot_ssvm | Success | 133.69 | test_ssvm.py
test_06_stop_cpvm | Success | 131.50 | test_ssvm.py
test_05_stop_ssvm | Success | 133.48 | test_ssvm.py
test_04_cpvm_internals | Success | 1.19 | test_ssvm.py
test_03_ssvm_internals | Success | 3.29 | test_ssvm.py
test_02_list_cpvm_vm | Success | 0.09 | test_ssvm.py
test_01_list_sec_storage_vm | Success | 0.09 | test_ssvm.py
test_01_snapshot_root_disk | Success | 11.01 | test_snapshots.py
test_04_change_offering_small | Success | 239.58 | test_service_offerings.py
test_03_delete_service_offering | Success | 0.03 | test_service_offerings.py
test_02_edit_service_offering | Success | 0.06 | test_service_offerings.py
test_01_create_service_offering | Success | 0.06 | test_service_offerings.py
test_02_sys_template_ready | Success | 0.09 | test_secondary_storage.py
test_01_sys_vm_start | Success | 0.12 | test_secondary_storage.py
test_09_reboot_router | Success | 35.23 | test_routers.py
test_08_start_router | Success | 30.20 | test_routers.py
test_07_stop_router | Success | 10.12 | test_routers.py
test_06_router_advanced | Success | 0.04 | test_routers.py
test_05_router_basic | Success | 0.03 | test_routers.py
test_04_restart_network_wo_cleanup | Success | 5.64 | test_routers.py
test_03_restart_network_cleanup | Success | 60.38 | test_routers.py
test_02_router_internal_adv | Success | 1.03 | test_routers.py
test_01_router_internal_basic | Success | 0.56 | test_routers.py
test_router_dns_guestipquery | Success | 76.61 | test_router_dns.py
test_router_dns_externalipquery | Success | 0.05 | test_router_dns.py
test_router_dhcphosts | Success | 386.94 | test_router_dhcphosts.py
te

[GitHub] cloudstack issue #1711: XenServer 7 Support

[blueorangutan]

Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1711
  
@rhtyd a Trillian-Jenkins test job (centos7 mgmt + xenserver-65sp1) has 
been kicked to run smoke tests


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack issue #1711: XenServer 7 Support

[rhtyd]

Github user rhtyd commented on the issue:

https://github.com/apache/cloudstack/pull/1711
  
@blueorangutan test centos7 xenserver-65sp1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #1840: CLOUDSTACK-9685: delete snapshot on primary a...

[anshul1886]

GitHub user anshul1886 opened a pull request:

https://github.com/apache/cloudstack/pull/1840

CLOUDSTACK-9685: delete snapshot on primary associated with a volume …

…when that volume is deleted

 as that snapshot will never be going to use again and also it will 
fill up primary storage

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/anshul1886/cloudstack-1 CLOUDSTACK-9685

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1840.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1840


commit 336df84f1787de962a67d0a34551f9027303040e
Author: Anshul Gangwar 
Date:   2016-12-20T09:47:08Z

CLOUDSTACK-9685: delete snapshot on primary associated with a volume when 
that volume is deleted
 as that snapshot will never be going to use again and also it will 
fill up primary storage




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: patchviasocket seems to be broken with qemu 2.3(+?)

[Wei ZHOU]

Hi Linas,

Good to know it. It looks increase the listening time in
/etc/init.d/cloud-early-config will fix it (default is 5 times * 2 seconds).

-Wei

2016-12-20 9:23 GMT+01:00 Linas Žilinskas :

> I don't think the issue is the same. As i mentioned in the original report
> and my findings afterwards, this is a specifically qemu issue which was
> fixed in 2.4.0-rc3.
>
> The issue was the way qemu exposes the socket to communicate with VM. It
> didn't queue data, so unless the VM was listening on /dev/vport.. at the
> time when data is sent, it would never reiceive it. 2.4.0-rc3 fixed this by
> queueing the data sent, so once sent, it was accessible (only once) when
> the VM checked /dev/vport..
>
> On 19/12/16 10:37, Wei ZHOU wrote:
>
> Hi Linas,
>
> It seems the issue you mentioned has been fixed by the commits for
> https://issues.apache.org/jira/browse/CLOUDSTACK-2823
>
> CloudStack-agent will try to pass the boot args 30 times if the console Ip
> is not accessible.
>
> Weird.
> -Wei
>
>

Re: patchviasocket seems to be broken with qemu 2.3(+?)

[Wei ZHOU]

Hi Synhrul,

Could you upload the /var/log/cloud.log ?

-Wei

2016-12-20 3:18 GMT+01:00 Syahrul Sazli Shaharir :

> On 2016-12-19 18:10, Syahrul Sazli Shaharir wrote:
>
>> On 2016-12-19 17:03, Linas Žilinskas wrote:
>>
>>> From the logs it doesn't seem that the script timeouts. "Execution is
>>> successful", so it manages to pass the data over the socket.
>>>
>>> I guess the systemvm just doesn't configure itself for some reason.
>>>
>>
>> You are right, I was able to enter the router VM console at some point
>> during the timeout loops, and able to capture syslog output during the
>> loop:-
>>
>> http://pastebin.com/n37aHeSa
>>
>
> I restarted another network, and that network's router VM was able to be
> recreated, even on the same host as the failed network (and both networks
> are exactly same configuration, only VLAN & subnet are different).
> Comparing between the two syslog outputs during boot shows the problematic
> network router VM self-configuration got stuck in vm_dhcp_entry.json .
>
> 1. Working network router VM : http://pastebin.com/Y6zpDa6M
> 2. Non-working network router VM : http://pastebin.com/jzfGMGQB
>
> Thanks.
>
>
>
>> Also, in my personal tests, I noticed some different behaviour with
>>> different kernels. Don't remember the specifics right now, but on some
>>> combinations (qemu / kernel) the socket acted differently. For example
>>> the data was sent over the socket, but wasn't visible inside the VM.
>>> Other times the socket would be stuck from the host side.
>>>
>>> So i would suggest testing different kernels (3.x, 4.4.x, 4.8.x) or
>>> try to login to the system vm and see what's happening from inside.
>>>
>>
>> Will do this next and feedback the results here.
>>
>> Thanks for your help! :)
>>
>>
>> On 12/16/16 03:46, Syahrul Sazli Shaharir wrote:
>>>
>>> On 2016-12-16 11:27, Syahrul Sazli Shaharir wrote:
 On Wed, 26 Oct 2016, Linas ?ilinskas wrote:

 So after some investigation I've found out that qemu 2.3.0 is indeed
 broken, at least the way CS uses the qemu chardev/socket.

 Not sure in which specific version it happened, but it was fixed in
 2.4.0-rc3, specifically noting that CloudStack 4.2 was not working.

 qemu git commit: 4bf1cb03fbc43b0055af60d4ff093d6894aa4338

 Also attaching the patch from that commit.

 For our own purposes i've included the patch to the qemu-kvm-ev
 package (2.3.0) and all is well.

 Hi,

 I am facing the exact same issue on latest Cloudstack 4.9.0.1, on
 latest CentOS 7.3.1611, with latest qemu-kvm-ev-2.6.0-27.1.el7
 package.

 The issue initially surfaced following a heartbeat-induced reset of
 all hosts, when it was on CS 4.8 @ CentOS 7.0 and stock
 qemu-kvm-1.5.3. Since then, the patchviasocket.pl/py timeouts
 persisted for 1 out of 4 router VM/networks, even after upgrading to

 latest code. (I have checked the qemu-kvm-ev-2.6.0-27.1.el7 source,
 and the patched code are pretty much still intact, as per the
 2.4.0-rc3 commit).

 Any help would be greatly appreciated.

 Thanks.

 (Attached are some debug logs from the host's agent.log)

>>>
>>> Here are the debug logs as mentioned: http://pastebin.com/yHdsMNzZ
>>>
>>> Thanks.
>>>
>>> --sazli

 On 2016-10-20 09:59, Linas ?ilinskas wrote:

 Hi.

 We have made an upgrade to 4.9.

 Custom build packages with our own patches, which in my mind (i'm
 the only
 one patching those) should not affect the issue i'll describe.

 I'm not sure whether we didn't notice it before, or it's actually
 related
 to something in 4.9

 Basically our system vm's were unable to be patched via the qemu
 socket.
 The script simply error'ed out with a timeout while trying to push
 the
 data to the socket.

 Executing it manually (with cmd line from the logs) resulted the
 same. I
 even tried the old perl variant, which also had same result.

 So finally we found out that this issue happens only on our HVs
 which run
 qemu 2.3.0, from the centos 7 special interest virtualization repo.
 Other
 ones that run qemu 1.5, from official repos, can patch the system
 vms
 fine.

 So i'm wondering if anyone tested 4.9 with kvm with qemu >= 2.x?
 Maybe it
 something else special in our setup. e.g. we're running the HVs
 from a
 preconfigured netboot image (pxe), but all of them, including those
 with
 qemu 1.5, so i have no idea.

 Linas ?ilinskas
 Head of Development
 website  [1] facebook
  [2] twitter
  [3] linkedin
 
 [4]

 Host1Plus is a division of Digital Energy Technologies Ltd.

 26 York Street, London W1U 6PZ, Unite

Re: patchviasocket seems to be broken with qemu 2.3(+?)

[Linas Žilinskas]

I don't think the issue is the same. As i mentioned in the original 
report and my findings afterwards, this is a specifically qemu issue 
which was fixed in 2.4.0-rc3.


The issue was the way qemu exposes the socket to communicate with VM. It 
didn't queue data, so unless the VM was listening on /dev/vport.. at the 
time when data is sent, it would never reiceive it. 2.4.0-rc3 fixed this 
by queueing the data sent, so once sent, it was accessible (only once) 
when the VM checked /dev/vport..



On 19/12/16 10:37, Wei ZHOU wrote:

Hi Linas,

It seems the issue you mentioned has been fixed by the commits for 
https://issues.apache.org/jira/browse/CLOUDSTACK-2823


CloudStack-agent will try to pass the boot args 30 times if the 
console Ip is not accessible.


Weird.

-Wei

2016-12-19 10:03 GMT+01:00 Linas Žilinskas >:


From the logs it doesn't seem that the script timeouts. "Execution
is successful", so it manages to pass the data over the socket.

I guess the systemvm just doesn't configure itself for some reason.

Also, in my personal tests, I noticed some different behaviour
with different kernels. Don't remember the specifics right now,
but on some combinations (qemu / kernel) the socket acted
differently. For example the data was sent over the socket, but
wasn't visible inside the VM. Other times the socket would be
stuck from the host side.

So i would suggest testing different kernels (3.x, 4.4.x, 4.8.x)
or try to login to the system vm and see what's happening from inside.


On 12/16/16 03:46, Syahrul Sazli Shaharir wrote:

On 2016-12-16 11:27, Syahrul Sazli Shaharir wrote:

On Wed, 26 Oct 2016, Linas ?ilinskas wrote:


So after some investigation I've found out that qemu 2.3.0 is
indeed broken, at least the way CS uses the qemu chardev/socket.

Not sure in which specific version it happened, but it was
fixed in 2.4.0-rc3, specifically noting that CloudStack 4.2 was
not working.

qemu git commit: 4bf1cb03fbc43b0055af60d4ff093d6894aa4338

Also attaching the patch from that commit.


For our own purposes i've included the patch to the qemu-kvm-ev
package (2.3.0) and all is well.


Hi,

I am facing the exact same issue on latest Cloudstack 4.9.0.1, on
latest CentOS 7.3.1611, with latest qemu-kvm-ev-2.6.0-27.1.el7
package.

The issue initially surfaced following a heartbeat-induced reset of
all hosts, when it was on CS 4.8 @ CentOS 7.0 and stock
qemu-kvm-1.5.3. Since then, the patchviasocket.pl/py
 timeouts
persisted for 1 out of 4 router VM/networks, even after
upgrading to
latest code. (I have checked the qemu-kvm-ev-2.6.0-27.1.el7 source,
and the patched code are pretty much still intact, as per the
2.4.0-rc3 commit).

Any help would be greatly appreciated.

Thanks.

(Attached are some debug logs from the host's agent.log)


Here are the debug logs as mentioned: http://pastebin.com/yHdsMNzZ

Thanks.



--sazli




On 2016-10-20 09:59, Linas ?ilinskas wrote:


 Hi.

 We have made an upgrade to 4.9.

 Custom build packages with our own patches, which in my mind
(i'm the only
 one patching those) should not affect the issue i'll describe.

 I'm not sure whether we didn't notice it before, or it's
actually related
 to something in 4.9

 Basically our system vm's were unable to be patched via the
qemu socket.
 The script simply error'ed out with a timeout while trying to
push the
 data to the socket.

 Executing it manually (with cmd line from the logs) resulted
the same. I
 even tried the old perl variant, which also had same result.

 So finally we found out that this issue happens only on our
HVs which run
 qemu 2.3.0, from the centos 7 special interest virtualization
repo. Other
 ones that run qemu 1.5, from official repos, can patch the
system vms
 fine.

 So i'm wondering if anyone tested 4.9 with kvm with qemu >=
2.x? Maybe it
 something else special in our setup. e.g. we're running the
HVs from a
 preconfigured netboot image (pxe), but all of them, including
those with
 qemu 1.5, so i have no idea.


 Linas ?ilinskas
 Head of Development
 website 
 facebook

 twitter

 linkedin




 Host1Plus is a division of Digital Energy Technologies Ltd.

 26 York Street, London W1U 6PZ, United Kingdom



Linas ?ilinskas
Head of Development
website