I mean we are populating the template just like we would do with normal NFS using the UI.
ACS takes care of pushing to swift. FG On Aug 26, 2014 6:02 AM, "Francois Gaudreault" <fgaudrea...@cloudops.com> wrote: > Punith, > > We are using Swift. We have a tmpauth proxy. > > FG > On Aug 26, 2014 2:48 AM, "Punith S" <punit...@cloudbyte.com> wrote: > >> sure mike, >> >> since i don't have a S3 account, i'm getting one today. >> >> francois, can you brief me how you seeded your templates into S3. >> >> thanks! >> >> >> On Mon, Aug 25, 2014 at 11:16 PM, Mike Tutkowski < >> mike.tutkow...@solidfire.com> wrote: >> >>> Yes, I expect we'll see the same issue with S3, as well. >>> >>> Punith - Is this something you might have time to investigate? Perhaps >>> Edison can point us in the right direction here. >>> >>> >>> On Mon, Aug 25, 2014 at 5:17 AM, Francois Gaudreault < >>> fgaudrea...@cloudops.com> wrote: >>> >>> > Punith, >>> > >>> > I highly anticipate the same issue with S3... it shares a lot of code >>> with >>> > swift. >>> > >>> > My focus would be swift, but we should fix for both :) >>> > >>> > FG >>> > On Aug 25, 2014 6:33 AM, "Punith S" <punit...@cloudbyte.com> wrote: >>> > >>> > > thanks for opening this thread mike, >>> > > >>> > > since i only use nfs as my secondary storage provider, i didn't see >>> this >>> > > issue till date. >>> > > >>> > > is this issue occurring even using a S3 secondary storage with >>> staging >>> > nfs >>> > > store ? >>> > > >>> > > if so like edison pointed we need to fetch the virtual size from the >>> nfs >>> > > store instead of S3 in the deploymentmanager. >>> > > >>> > > thanks >>> > > >>> > > >>> > > On Sat, Aug 23, 2014 at 3:45 AM, Mike Tutkowski < >>> > > mike.tutkow...@solidfire.com> wrote: >>> > > >>> > > > Hey Edison, >>> > > > >>> > > > Do you know how difficult/easy of a fix this is, who might be >>> available >>> > > to >>> > > > put this fix in, and for what release (hopefully 4.4.1) this fix >>> could >>> > > find >>> > > > its way in? >>> > > > >>> > > > Thanks! >>> > > > Mike >>> > > > >>> > > > >>> > > > On Fri, Aug 22, 2014 at 3:37 PM, Francois Gaudreault < >>> > > > fgaudrea...@cloudops.com> wrote: >>> > > > >>> > > > > Min, >>> > > > > >>> > > > > Ok, but this is not the behavior I see. Even without requesting >>> a VM >>> > > > > create, the template is pushed to the staging NFS at least once. >>> Is >>> > it >>> > > > > downloaded there or pushed after download, that I am not sure. I >>> was >>> > > > > assuming the swift upload bash script was executed after the >>> template >>> > > is >>> > > > on >>> > > > > the staging. >>> > > > > >>> > > > > Anyway... the focus is on the virt size, and you all know the >>> code >>> > > better >>> > > > > than I do :) >>> > > > > >>> > > > > FG >>> > > > > On Aug 22, 2014 5:28 PM, "Min Chen" <min.c...@citrix.com> wrote: >>> > > > > >>> > > > >> No. For S3/Swift, register template will directly upload >>> templates >>> > to >>> > > S3 >>> > > > >> without going through staging NFS. It will only be copied to >>> staging >>> > > NFS >>> > > > >> when we first use that template to provision a VM. >>> > > > >> >>> > > > >> Thanks >>> > > > >> -min >>> > > > >> >>> > > > >> On 8/22/14 2:25 PM, "Francois Gaudreault" < >>> fgaudrea...@cloudops.com >>> > > >>> > > > >> wrote: >>> > > > >> >>> > > > >> >Edison, >>> > > > >> > >>> > > > >> >Isnt the templates downloaded to the Staging NFS first? >>> > > > >> > >>> > > > >> >FG >>> > > > >> >On Aug 22, 2014 5:20 PM, "Edison Su" <edison...@citrix.com> >>> wrote: >>> > > > >> > >>> > > > >> >> I know the reason why the size of template doesn¹t have >>> correct >>> > > > virtual >>> > > > >> >> size if it¹s registered in S3/Swift: >>> > > > >> >> In case of s3/swift, the template is directly stored into >>> > s3/swift >>> > > > >> >>through >>> > > > >> >> swift/s3 api, there is no place for cloudstack to look into >>> > > template, >>> > > > >> to >>> > > > >> >> find out the virtual size during template registration. >>> > > > >> >> While, if secondary storage is NFS, the template is first >>> stored >>> > on >>> > > > >> >> NFS(which has file system), cloudstack can unzip the >>> template(if >>> > > > it¹s a >>> > > > >> >> zipped file), and read virtual size from the file, then >>> report >>> > back >>> > > > to >>> > > > >> >>mgt >>> > > > >> >> server. >>> > > > >> >> In order to fix it, we can add some code as: all the >>> templates >>> > > > >> >>registered >>> > > > >> >> on Swift/S3, need to be downloaded to a NFS intermediate >>> storage >>> > > > before >>> > > > >> >>it >>> > > > >> >> can be consumed by primary storage. After the download >>> finished, >>> > > then >>> > > > >> we >>> > > > >> >> check virtual size of template, and report back to mgt >>> > > server/update >>> > > > DB >>> > > > >> >>etc. >>> > > > >> >> >>> > > > >> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com] >>> > > > >> >> Sent: Friday, August 22, 2014 1:38 PM >>> > > > >> >> To: dev@cloudstack.apache.org >>> > > > >> >> Cc: Edison Su >>> > > > >> >> Subject: S3/Swift Problem around Virtual Size >>> > > > >> >> >>> > > > >> >> Hi, >>> > > > >> >> >>> > > > >> >> This was brought up in a different e-mail thread, but I >>> wanted to >>> > > > make >>> > > > >> >>it >>> > > > >> >> more clear that it's related to CloudStack's download code >>> around >>> > > > >> >>S3/Swift, >>> > > > >> >> so I'm opening up a new thread. >>> > > > >> >> >>> > > > >> >> Francois (from CloudOps) noticed today that when he >>> downloaded a >>> > > > >> >>template >>> > > > >> >> (VHD format) to Swift (but it looks like the same applies >>> for S3) >>> > > > that >>> > > > >> >>the >>> > > > >> >> physical and virtual sizes are set to be the same. >>> > > > >> >> >>> > > > >> >> This appears to have the following consequence: >>> > > > >> >> >>> > > > >> >> You can download a template with a physical size of, say, 3 >>> GB >>> > and >>> > > a >>> > > > >> >>root >>> > > > >> >> disk that's supposed to be, say, 20 GB. Instead of the >>> virtual >>> > size >>> > > > >> >>showing >>> > > > >> >> as 20 GB, it shows as 3 GB. >>> > > > >> >> >>> > > > >> >> This is not an issue with NFS. In that situation, the two >>> sizes >>> > are >>> > > > >> >> correctly accounted for. >>> > > > >> >> >>> > > > >> >> What later can happen is the template is downloaded from >>> Swift >>> > and >>> > > > >> takes >>> > > > >> >> up an unexpected amount of space on the XenServer storage >>> > > repository >>> > > > >> >>(SR). >>> > > > >> >> >>> > > > >> >> If there is enough space on the SR, this isn't too big of a >>> deal. >>> > > > >> >>However, >>> > > > >> >> for so-called managed storage plug-ins (examples are >>> SolidFire >>> > and >>> > > > >> >> CloudByte), this will lead to them dynamically creating a SAN >>> > > volume >>> > > > of >>> > > > >> >>the >>> > > > >> >> wrong size. >>> > > > >> >> >>> > > > >> >> Francois opened up the following ticket: >>> > > > >> >> >>> > > > >> >> https://issues.apache.org/jira/browse/CLOUDSTACK-7406 >>> > > > >> >> >>> > > > >> >> Thanks! >>> > > > >> >> >>> > > > >> >> -- >>> > > > >> >> Mike Tutkowski >>> > > > >> >> Senior CloudStack Developer, SolidFire Inc. >>> > > > >> >> e: mike.tutkow...@solidfire.com<mailto: >>> > > mike.tutkow...@solidfire.com> >>> > > > >> >> o: 303.746.7302 >>> > > > >> >> Advancing the way the world uses the cloud< >>> > > > >> >> http://solidfire.com/solution/overview/?video=play> >>> > > > >> >> >>> > > > >> >>> > > > >> >>> > > > >>> > > > >>> > > > -- >>> > > > *Mike Tutkowski* >>> > > > *Senior CloudStack Developer, SolidFire Inc.* >>> > > > e: mike.tutkow...@solidfire.com >>> > > > o: 303.746.7302 >>> > > > Advancing the way the world uses the cloud >>> > > > <http://solidfire.com/solution/overview/?video=play>*™* >>> > > > >>> > > >>> > > >>> > > >>> > > -- >>> > > regards, >>> > > >>> > > punith s >>> > > cloudbyte.com >>> > > >>> > >>> >>> >>> >>> -- >>> *Mike Tutkowski* >>> *Senior CloudStack Developer, SolidFire Inc.* >>> e: mike.tutkow...@solidfire.com >>> o: 303.746.7302 >>> Advancing the way the world uses the cloud >>> <http://solidfire.com/solution/overview/?video=play>*™* >>> >> >> >> >> -- >> regards, >> >> punith s >> cloudbyte.com >> >
