[
https://issues.apache.org/jira/browse/CLOUDSTACK-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14909792#comment-14909792
]
Rajani Karuturi commented on CLOUDSTACK-8808:
---------------------------------------------
We have multiple file processors to process different types of image formats.
The processor interface has two methods getVirtualSize() and process().
* getVirtualSize() as the name says, returns the virtual size of the file and
is used at get the size while copying files from NFS to s3
* process() returns FormatInfo struct which has fileType, size, virutalSize,
filename.
on successfully downloading a template, each file is passed to all the
processors.process() and whichever returns a FormatInfo, that will be used to
create template.properties file.
If process() throws an InternalErrorException, template installation fails.
But, if process() returns null, template registration is successful with
template.properties missing some attributes like virtualSize, file format etc.
which results in this bug on restart of ssvm/cloud service/management server.
processors and the current way of handling exceptions in getVirtualSize() and
process()
||processor||getVirtualSize() success||getVirtualSize() failure||process() if
file found||
|IsoProcessor|actual size|no failure. always returns actual size|always returns
info with virtual size|
|OVAProcessor|virtual size| actual size|returns null incase untar fails. else
info with virtual size|
|QCOW2Processor| virtual size| IOException|null incase of IOException from
getvirtualsize()|
|RawImageProcessor|actual size|no failure. always returns actual size|always
returns info with virtual size|
|TARProcessor|actual size|no failure. always returns actual size|always returns
info with virtual size|
|VHDProcessor|virtual size| IOException|null incase of IOException from
getvirtualsize()|
|VmdkProcessor|virtual size| actual size|InternalErrorException incase virtual
size calculation fails|
I am planning to make the below changes:
getVirtualSize() to always return size(if it can calculate, get virtual size
else return file size). This would mean the following changes
# QCOW2Processor.getVirtualSize() to return file size if virtual size
calculation fails
# VHDProcessor.getVirtualSize() to return file size if virtual size calculation
fails
process() to throw InternalErrorException if virtual size calculation fails or
any other exceptions occur. This would mean the following changes
# OVAProcessor to throw InternalErrorException if untar fails
# QCOW2Processor to throw InternalErrorException if virtual size calculation
fails
# VHDProcessor to throw InternalErrorException if virtual size calculation fails
> Successfully registered VHD template is downloaded again due to missing
> virtualsize property in template.properties
> -------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-8808
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8808
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Secondary Storage
> Affects Versions: 4.4.4, 4.6.0
> Environment: Seen on NFS as sec storage
> Reporter: Remi Bergsma
> Assignee: Rajani Karuturi
> Priority: Blocker
>
> We noticed all of our templates are downloaded again as soon as we restart
> SSVM, its Cloud service or the management server it connects to.
> A scan done by the SSVM (listvmtmplt.sh) returns the template, but it is
> rejected later (Post download installation was not completed) because (Format
> is invalid) due to missing virtualSize property in template.properties.
> The initial registration did succeed however. I'd either want the
> registration to fail, or it to succeed. Not first succeed (and spin VMs
> without a problem) then fail unexpectedly later.
> This is the script processing the download:
> services/secondary-storage/server/src/org/apache/cloudstack/storage/template/DownloadManagerImpl.java
> 759 private List<String> listTemplates(String rootdir) {
>
> 760 List<String> result = new ArrayList<String>();
>
> 761
>
> 762 Script script = new Script(listTmpltScr, s_logger);
>
> 763 script.add("-r", rootdir);
> For example this becomes:
> ==> /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r
> /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0
> In this log file, it processes the output:
> less /var/log/cloud/cloud.out
> 2015-09-04 08:39:54,622 WARN [storage.template.DownloadManagerImpl]
> (agentRequest-Handler-1:null) Post download installation was not completed
> for /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/2/1607
> This error message is generated here:
> services/secondary-storage/server/src/org/apache/cloudstack/storage/template/DownloadManagerImpl.java
>
> 780 List<String> publicTmplts = listTemplates(templateDir);
>
> 781 for (String tmplt : publicTmplts) {
>
> 782 String path = tmplt.substring(0,
> tmplt.lastIndexOf(File.separator));
> 783 TemplateLocation loc = new TemplateLocation(_storage, path);
>
> 784 try {
>
> 785 if (!loc.load()) {
>
> 786 s_logger.warn("Post download installation was not
> completed for " + path);
> 787 // loc.purge();
>
> 788 _storage.cleanup(path, templateDir);
>
> 789 continue;
>
> 790 }
>
> 791 } catch (IOException e) {
>
> 792 s_logger.warn("Unable to load template location " +
> path, e);
> 793 continue;
>
> 794 }
> In the logs this message is also seen:
> MCCP-ADMIN-1|s-32436-VM CLOUDSTACK: 10:09:17,333 WARN TemplateLocation:196 -
> Format is invalid
> It is generated here:
> .//core/src/com/cloud/storage/template/TemplateLocation.java
> 192 public boolean addFormat(FormatInfo newInfo) {
>
> 193 deleteFormat(newInfo.format);
>
> 194
>
> 195 if (!checkFormatValidity(newInfo)) {
>
> 196 s_logger.warn("Format is invalid ");
>
> 197 return false;
>
> 198 }
>
> 199
>
> 200 _props.setProperty("virtualsize",
> Long.toString(newInfo.virtualSize));
> 201 _formats.add(newInfo);
>
> 202 return true;
>
> 203 }
> This returns false if checkFormatValidity is false.
> checkFormatValidity:
> 209 protected boolean checkFormatValidity(FormatInfo info) {
>
> 210 return (info.format != null && info.size > 0 && info.virtualSize
> > 0 && info.filename != null);
> 211 }
> This returns false if virtualSize is missing in template.properties. And
> indeed it is missing.
> Examples:
> Working KVM
> root@s-44134-VM:/mnt/SecStorage/884db36b-cd00-3cf0-b812-831e9e50a4b3/template/tmpl/2/2932#
> cat template.properties
> #
> #Thu Sep 03 11:41:44 UTC 2015
> filename=125672aa-62c0-30f1-ae1b-153486aa10eb.qcow2
> id=2932
> qcow2.size=1808203776
> public=true
> uniquename=2932-2-5c53bbe1-0c32-32a9-a82c-a5eefcb472e5
> qcow2.virtualsize=21474836480
> virtualsize=21474836480
> checksum=4c28fc94a7a9cbe858918a97f3b78294
> hvm=true
> description=Centos7-x86_64-Sbp_cis-KVM release 2015-35 build
> qcow2=true
> qcow2.filename=125672aa-62c0-30f1-ae1b-153486aa10eb.qcow2
> Working XenServer
> cat
> /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/9/603/template.properties
> filename=16447323-ae24-4956-ac67-316eed8a2626.vhd
> vhd=true
> id=603
> vhd.filename=16447323-ae24-4956-ac67-316eed8a2626.vhd
> public=false
> uniquename=16447323-ae24-4956-ac67-316eed8a2626
> vhd.virtualsize=107374182400
> virtualsize=107374182400
> checksum=
> hvm=true
> description=570a3650f0-c982-334a-81b1-e8cf961363e4
> vhd.size=69028311552
> size=69028311552
> Not working XenServer
> root@s-32435-VM:/mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/2/1607#
> cat template.properties
> #
> #Fri Sep 04 09:05:24 UTC 2015
> filename=39794796-3f92-3168-9c4e-a64da5ee06e8.vhd
> uniquename=1607-2-0a13984d-6724-3b9f-8bd1-2798ef859167
> size=21475270656
> checksum=6eb85d82cb18032eba0cc8fe8a84a583
> description=Centos7-x86_64-Sbp_cis-XenServer release 2015-35 build
> hvm=true
> public=true
> id=1607
> Confirmed the virtualsize is missing. But why is this successfully registered
> in the first place?
> Steps to reproduce:
> - register template
> - have it install successfully
> - check templates.properties file
> sometimes virtualsize is missing
> - if it's there, remove it
> - restart cloud service on SSVM
> - the download will start again
> in my case virtualsize is still missing
> - this becomes an endless loop.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
