Punith, We are using Swift. We have a tmpauth proxy.
FG On Aug 26, 2014 2:48 AM, "Punith S" <punit...@cloudbyte.com> wrote: > sure mike, > > since i don't have a S3 account, i'm getting one today. > > francois, can you brief me how you seeded your templates into S3. > > thanks! > > > On Mon, Aug 25, 2014 at 11:16 PM, Mike Tutkowski < > mike.tutkow...@solidfire.com> wrote: > >> Yes, I expect we'll see the same issue with S3, as well. >> >> Punith - Is this something you might have time to investigate? Perhaps >> Edison can point us in the right direction here. >> >> >> On Mon, Aug 25, 2014 at 5:17 AM, Francois Gaudreault < >> fgaudrea...@cloudops.com> wrote: >> >> > Punith, >> > >> > I highly anticipate the same issue with S3... it shares a lot of code >> with >> > swift. >> > >> > My focus would be swift, but we should fix for both :) >> > >> > FG >> > On Aug 25, 2014 6:33 AM, "Punith S" <punit...@cloudbyte.com> wrote: >> > >> > > thanks for opening this thread mike, >> > > >> > > since i only use nfs as my secondary storage provider, i didn't see >> this >> > > issue till date. >> > > >> > > is this issue occurring even using a S3 secondary storage with staging >> > nfs >> > > store ? >> > > >> > > if so like edison pointed we need to fetch the virtual size from the >> nfs >> > > store instead of S3 in the deploymentmanager. >> > > >> > > thanks >> > > >> > > >> > > On Sat, Aug 23, 2014 at 3:45 AM, Mike Tutkowski < >> > > mike.tutkow...@solidfire.com> wrote: >> > > >> > > > Hey Edison, >> > > > >> > > > Do you know how difficult/easy of a fix this is, who might be >> available >> > > to >> > > > put this fix in, and for what release (hopefully 4.4.1) this fix >> could >> > > find >> > > > its way in? >> > > > >> > > > Thanks! >> > > > Mike >> > > > >> > > > >> > > > On Fri, Aug 22, 2014 at 3:37 PM, Francois Gaudreault < >> > > > fgaudrea...@cloudops.com> wrote: >> > > > >> > > > > Min, >> > > > > >> > > > > Ok, but this is not the behavior I see. Even without requesting a >> VM >> > > > > create, the template is pushed to the staging NFS at least once. >> Is >> > it >> > > > > downloaded there or pushed after download, that I am not sure. I >> was >> > > > > assuming the swift upload bash script was executed after the >> template >> > > is >> > > > on >> > > > > the staging. >> > > > > >> > > > > Anyway... the focus is on the virt size, and you all know the code >> > > better >> > > > > than I do :) >> > > > > >> > > > > FG >> > > > > On Aug 22, 2014 5:28 PM, "Min Chen" <min.c...@citrix.com> wrote: >> > > > > >> > > > >> No. For S3/Swift, register template will directly upload >> templates >> > to >> > > S3 >> > > > >> without going through staging NFS. It will only be copied to >> staging >> > > NFS >> > > > >> when we first use that template to provision a VM. >> > > > >> >> > > > >> Thanks >> > > > >> -min >> > > > >> >> > > > >> On 8/22/14 2:25 PM, "Francois Gaudreault" < >> fgaudrea...@cloudops.com >> > > >> > > > >> wrote: >> > > > >> >> > > > >> >Edison, >> > > > >> > >> > > > >> >Isnt the templates downloaded to the Staging NFS first? >> > > > >> > >> > > > >> >FG >> > > > >> >On Aug 22, 2014 5:20 PM, "Edison Su" <edison...@citrix.com> >> wrote: >> > > > >> > >> > > > >> >> I know the reason why the size of template doesn¹t have >> correct >> > > > virtual >> > > > >> >> size if it¹s registered in S3/Swift: >> > > > >> >> In case of s3/swift, the template is directly stored into >> > s3/swift >> > > > >> >>through >> > > > >> >> swift/s3 api, there is no place for cloudstack to look into >> > > template, >> > > > >> to >> > > > >> >> find out the virtual size during template registration. >> > > > >> >> While, if secondary storage is NFS, the template is first >> stored >> > on >> > > > >> >> NFS(which has file system), cloudstack can unzip the >> template(if >> > > > it¹s a >> > > > >> >> zipped file), and read virtual size from the file, then report >> > back >> > > > to >> > > > >> >>mgt >> > > > >> >> server. >> > > > >> >> In order to fix it, we can add some code as: all the templates >> > > > >> >>registered >> > > > >> >> on Swift/S3, need to be downloaded to a NFS intermediate >> storage >> > > > before >> > > > >> >>it >> > > > >> >> can be consumed by primary storage. After the download >> finished, >> > > then >> > > > >> we >> > > > >> >> check virtual size of template, and report back to mgt >> > > server/update >> > > > DB >> > > > >> >>etc. >> > > > >> >> >> > > > >> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com] >> > > > >> >> Sent: Friday, August 22, 2014 1:38 PM >> > > > >> >> To: dev@cloudstack.apache.org >> > > > >> >> Cc: Edison Su >> > > > >> >> Subject: S3/Swift Problem around Virtual Size >> > > > >> >> >> > > > >> >> Hi, >> > > > >> >> >> > > > >> >> This was brought up in a different e-mail thread, but I >> wanted to >> > > > make >> > > > >> >>it >> > > > >> >> more clear that it's related to CloudStack's download code >> around >> > > > >> >>S3/Swift, >> > > > >> >> so I'm opening up a new thread. >> > > > >> >> >> > > > >> >> Francois (from CloudOps) noticed today that when he >> downloaded a >> > > > >> >>template >> > > > >> >> (VHD format) to Swift (but it looks like the same applies for >> S3) >> > > > that >> > > > >> >>the >> > > > >> >> physical and virtual sizes are set to be the same. >> > > > >> >> >> > > > >> >> This appears to have the following consequence: >> > > > >> >> >> > > > >> >> You can download a template with a physical size of, say, 3 GB >> > and >> > > a >> > > > >> >>root >> > > > >> >> disk that's supposed to be, say, 20 GB. Instead of the virtual >> > size >> > > > >> >>showing >> > > > >> >> as 20 GB, it shows as 3 GB. >> > > > >> >> >> > > > >> >> This is not an issue with NFS. In that situation, the two >> sizes >> > are >> > > > >> >> correctly accounted for. >> > > > >> >> >> > > > >> >> What later can happen is the template is downloaded from Swift >> > and >> > > > >> takes >> > > > >> >> up an unexpected amount of space on the XenServer storage >> > > repository >> > > > >> >>(SR). >> > > > >> >> >> > > > >> >> If there is enough space on the SR, this isn't too big of a >> deal. >> > > > >> >>However, >> > > > >> >> for so-called managed storage plug-ins (examples are SolidFire >> > and >> > > > >> >> CloudByte), this will lead to them dynamically creating a SAN >> > > volume >> > > > of >> > > > >> >>the >> > > > >> >> wrong size. >> > > > >> >> >> > > > >> >> Francois opened up the following ticket: >> > > > >> >> >> > > > >> >> https://issues.apache.org/jira/browse/CLOUDSTACK-7406 >> > > > >> >> >> > > > >> >> Thanks! >> > > > >> >> >> > > > >> >> -- >> > > > >> >> Mike Tutkowski >> > > > >> >> Senior CloudStack Developer, SolidFire Inc. >> > > > >> >> e: mike.tutkow...@solidfire.com<mailto: >> > > mike.tutkow...@solidfire.com> >> > > > >> >> o: 303.746.7302 >> > > > >> >> Advancing the way the world uses the cloud< >> > > > >> >> http://solidfire.com/solution/overview/?video=play> >> > > > >> >> >> > > > >> >> > > > >> >> > > > >> > > > >> > > > -- >> > > > *Mike Tutkowski* >> > > > *Senior CloudStack Developer, SolidFire Inc.* >> > > > e: mike.tutkow...@solidfire.com >> > > > o: 303.746.7302 >> > > > Advancing the way the world uses the cloud >> > > > <http://solidfire.com/solution/overview/?video=play>*™* >> > > > >> > > >> > > >> > > >> > > -- >> > > regards, >> > > >> > > punith s >> > > cloudbyte.com >> > > >> > >> >> >> >> -- >> *Mike Tutkowski* >> *Senior CloudStack Developer, SolidFire Inc.* >> e: mike.tutkow...@solidfire.com >> o: 303.746.7302 >> Advancing the way the world uses the cloud >> <http://solidfire.com/solution/overview/?video=play>*™* >> > > > > -- > regards, > > punith s > cloudbyte.com >
