Hi Ilya,
Thank you for pointing that out. Is it possible for you to send me the
shell script first so I can at least register my templates and ISOs first
for me to continue testing? Any ETA on when the patch would be out?
I cannot proceed to test creating VM instances and perform further tests
without any templates being installed.
Looking forward to your reply, thank you.
Cheers.
On Sat, May 14, 2016 at 1:42 PM, ilya musayev
wrote:
> Indra,
>
> While I see there is a solution to your problem due to cloudstack changes
> with web servers, there is also another tangent to this issue..
>
> Let me know if your issue is resolved by leveraging a different web server
> for image downloads.
>
> But if not, or someone else comes across this message thread, here is a
> another tangent to this problem.
>
>
> Below is a summary to the problem we've seen that seem very similar
>
> In the nutshell, here is what we've seen and what we believe cause the
> issue.
>
> CloudStack Java Agent initiates 2 https sessions when attempting to
> download.
>
> 1) Session is established to web server to get the header information and
> keep it open..
> 2) Session is established to download the content - this is usually a
> longer stream operation
>
>
> What we've seen happen, as session # 2 goes on downloading, somehow http
> session #1 gets corrupted. Session #2 is eventually terminated as it
> completes the operation, yet session #1 remains to be open.
>
> This defies the logic in a way, because no packets are sent over the open
> session between SSVM and HTTP server. I'd expect for one of two parties
> sending termination packet - but that never happens. As the result - the
> 1st session hangs and prevents agent from proceed to the next stage of
> installing the template. I've seen the socket being open for days on SSVM
> and yet not a single packet comes - not even keep alive.
>
> Patch should be out soon. Credit goes to Marcus for figuring this out.
>
>
> I also have a shell script that can resolve the specific stuck template
> download, but i've made it geared toward QCOW2. I can post it if need be,
> but we should probably release a proper fix..
>
> Regards
> ilya
>
>
>
>
>
> On Wed, May 11, 2016 at 4:37 AM, Indra Pramana wrote:
>
> > Dear all,
> >
> > I setup a test environment using CloudStack 4.8.0 (latest from
> repository),
> > one management server and two KVM agent hosts running Ubuntu 14.04 LTS.
> > Setup a zone with basic networking because the test environment doesn't
> > support VLAN and the test management server only have 1 NIC. The private
> > and public network are using the same subnet. Using Ceph RBD as primary
> > storage and NFS as secondary storage.
> >
> > Zone is created successfully, the two hosts are added successfully and
> > connected to the management servers. SSVM and CPVM are created without
> any
> > issues. However, the default
> > "CentOS 5.5(64-bit) no GUI (KVM)" template is stuck at "Installing
> > Template" status and Ready = "No" although it seems to be downloaded
> > successfully. I also tried registering an Ubuntu ISO and it faces the
> same
> > problem, it's downloaded successfully but then will stuck at "Installing
> > ISO" status.
> >
> > I ran the SSVM health check and everything is OK. Also go through the
> > troubleshooting steps as per below documentation but still unable to
> > determine the root cause:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM,+templates,+Secondary+storage+troubleshooting
> >
> > I tried restarting the SSVM service, stop and start the SSVM and
> restarting
> > the management-server service, it will cause the download of the ISO and
> > template to be restarted, but it will then stuck at installing again.
> >
> > Logs from management server shows below lines which keeps on appearing /
> > looping every several seconds:
> >
> > ===
> > 2016-05-11 19:31:29,031 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > (Timer-6:ctx-369bfc57) (logid:0b1754a2) getCommandHostDelegation: class
> > org.apache.cloudstack.storage.command.DownloadProgressCommand
> > 2016-05-11 19:31:29,031 DEBUG [c.c.h.XenServerGuru]
> (Timer-6:ctx-369bfc57)
> > (logid:0b1754a2) getCommandHostDelegation: class
> > org.apache.cloudstack.storage.command.DownloadProgressCommand
> > 2016-05-11 19:31:29,031 DEBUG [o.a.c.s.RemoteHostEndPoint]
> > (Timer-6:ctx-369bfc57) (logid:0b1754a2) Sending command
> > org.apache.cloudstack.storage.command.DownloadProgressCommand to host: 3
> > 2016-05-11 19:31:29,034 DEBUG [c.c.a.t.Request] (Timer-6:ctx-369bfc57)
> > (logid:0b1754a2) Seq 3-3028107799453237894: Sending { Cmd , MgmtId:
> > 181122461655966, via: 3(s-3-VM), Ver: v1, Flags: 100011,
> > [{"org.apache.cloudstack.storage.command
> >
> >
> .DownloadProgressCommand":{"jobId":"cab4ada2-e52f-4c25-9c30-9ec23dc3fcd4","request":"GET_STATUS","hvm":false,"description":"CentOS
> > 5.5(64-bit) no GUI
> >
> >
>