forgot to cc myself
-------- Original message --------
From: Taylor <tschnei...@live.com>
Date: 7/19/17 08:08 (GMT-06:00)
To: users@cloudstack.apache.org
Subject: RE: Cloudstack 4.9 CentOS Template failure - Connection Reset
Hey Dag,
I have tried both. The health check is good and the vm behavior does not change
after recreation.
I think the problem is the network latency or the vm's resource allocation.
The download will work but a connection timeout occurs every 20MB so it needs
to be done in pieces. The hypervisors which hosts the vm is able to download
without a problem.
I am on a 100mbps switch. In the past I was using a 1000mbps.
Any other thoughts on debug or work around?
I think adding retry logic should be a simple fix?
-------- Original message --------
From: Dag Sonstebo <dag.sonst...@shapeblue.com>
Date: 7/19/17 03:11 (GMT-06:00)
To: users@cloudstack.apache.org
Subject: Re: Cloudstack 4.9 CentOS Template failure - Connection Reset
Hi Taylor,
This is most likely an issue with your environment rather than a bug. Take a
look at your public network and how that is connected to the internet. You have
to let CloudStack pull down the template, it’s difficult to manually populate
this.
A couple of other things to try:
- recreate the SSVM – simply delete it and CloudStack will generate a new one.
- from internally in the SSVM you can also run the SSVM check script, which
will do some basic health checks for you:
root@s-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh
================================================
First DNS server is 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 48 data bytes
56 bytes from 8.8.8.8: icmp_seq=0 ttl=53 time=24.146 ms
56 bytes from 8.8.8.8: icmp_seq=1 ttl=53 time=22.320 ms
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 22.320/23.233/24.146/0.913 ms
Good: Can ping DNS server
================================================
Good: DNS resolves download.cloud.com
================================================
nfs is currently mounted
Mount point is /mnt/SecStorage/a833f5f1-1c6d-3e54-9a55-1fc9b7875c54
Good: Can write to mount point
================================================
Management server is 10.10.45.2. Checking connectivity.
Good: Can connect to management server port 8250
================================================
Good: Java process is running
================================================
Tests Complete. Look for ERROR or WARNING above.
Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue
On 19/07/2017, 05:29, "Taylor" <tschnei...@live.com> wrote:
Hello,
I am experiencing an issue while trying to download the CentOS template.
It seems the connection is timing out and then failing.
To debug I logged into the SSVM and tried running a wget from the nfs
directory mounted on that vm. This also failed due to connection reset.
Wget will eventually succeed if i use retry logic as follows:
wget http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
--read-timeout=10
Using this setting the download will complete in pieces after multiple
timeouts (logs below).
Is there a work around to add retry logic? Can I manually download and add
the template to the database and restart the service? How can I file a bug
report?
Thanks,
Taylor
=============================================================================
LOGS:
=============================================================================
root@s-103-VM:/mnt/SecStorage/5d8e791e-01cc-3d7c-84d8-f469944056e0/template/tmpl/1/5#
wget http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
--read-timeout=10
--2017-07-19 03:25:10--
http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Resolving download.cloud.com (download.cloud.com)... 54.231.81.40
Connecting to download.cloud.com (download.cloud.com)|54.231.81.40|:80...
connected.
HTTP request sent, awaiting response... 200 OK
Length: 374730926 (357M) [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
19% [============================>
]
74,734,204 --.-K/s in 97s
2017-07-19 03:26:55 (756 KB/s) - Read error at byte 74734204/374730926
(Connection timed out). Retrying.
--2017-07-19 03:26:56-- (try: 2)
http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|54.231.81.40|:80...
failed: Connection timed out.
Resolving download.cloud.com (download.cloud.com)... 52.216.81.16
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80...
connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 299996722 (286M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
24% [+++++++++++++++++++++++++++++======>
]
93,094,824 --.-K/s in 24s
2017-07-19 03:28:23 (740 KB/s) - Read error at byte 93094824/374730926
(Connection timed out). Retrying.
--2017-07-19 03:28:25-- (try: 3)
http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80...
connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 281636102 (269M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
29% [++++++++++++++++++++++++++++++++++++======>
]
112,111,876 --.-K/s in 26s
2017-07-19 03:29:23 (702 KB/s) - Read error at byte 112111876/374730926
(Connection timed out). Retrying.
--2017-07-19 03:29:26-- (try: 4)
http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80...
connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 262619050 (250M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
34% [+++++++++++++++++++++++++++++++++++++++++++=======>
]
130,790,615 --.-K/s in 26s
2017-07-19 03:30:23 (712 KB/s) - Read error at byte 130790615/374730926
(Connection timed out). Retrying.
--2017-07-19 03:30:27-- (try: 5)
http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80...
connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 243940311 (233M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
49%
[+++++++++++++++++++++++++++++++++++++++++++++++++++====================>
]
185,802,362 --.-K/s in 49s
2017-07-19 03:31:23 (1.08 MB/s) - Read error at byte 185802362/374730926
(Connection timed out). Retrying.
--2017-07-19 03:31:28-- (try: 6)
http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.81.16|:80...
failed: Connection timed out.
Resolving download.cloud.com (download.cloud.com)... 52.216.225.48
Connecting to download.cloud.com (download.cloud.com)|52.216.225.48|:80...
connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 188928564 (180M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
60%
[++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++================>
] 226,947,877 --.-K/s
in 32s
2017-07-19 03:33:11 (1.21 MB/s) - Read error at byte 226947877/374730926
(Connection timed out). Retrying.
--2017-07-19 03:33:17-- (try: 7)
http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.225.48|:80...
connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 147783049 (141M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
65%
[+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++======>
] 247,259,880 --.-K/s in
28s
2017-07-19 03:33:46 (697 KB/s) - Read error at byte 247259880/374730926
(Connection timed out). Retrying.
--2017-07-19 03:33:53-- (try: 8)
http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2
Connecting to download.cloud.com (download.cloud.com)|52.216.225.48|:80...
connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 374730926 (357M), 127471046 (122M) remaining [binary/octet-stream]
Saving to: `centos56-x86_64.vhd.bz2'
100%[++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++==================================================>]
374,730,926 1010K/s in 2m 10s
dag.sonst...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
