Re: Merging/Removing Secondary Storage NFS
t;>> vhd=true >> >>>>> id=5 >> >>>>> vhd.filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd >> >>>>> public=true >> >>>>> uniquename=centos56-x86_64-xen >> >>>>> vhd.virtualsize=21474836480 >> >>>>> virtualsize=21474836480 >> >>>>> checksum=905cec879afd9c9d22ecc8036131a180 >> >>>>> hvm=false >> >>>>> description=CentOS 5.6(64-bit) no GUI (XenServer) >> >>>>> vhd.size=1758786048 >> >>>>> size=1758786048 >> >>>>> >> >>>>> S1 template.properties file >> >>>>> # >> >>>>> #Wed Nov 21 23:14:59 UTC 2012 >> >>>>> filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd >> >>>>> vhd=true >> >>>>> id=5 >> >>>>> vhd.filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd >> >>>>> public=true >> >>>>> uniquename=centos56-x86_64-xen >> >>>>> vhd.virtualsize=21474836480 >> >>>>> virtualsize=21474836480 >> >>>>> checksum=905cec879afd9c9d22ecc8036131a180 >> >>>>> hvm=false >> >>>>> description=CentOS 5.6(64-bit) no GUI (XenServer) >> >>>>> vhd.size=1758786048 >> >>>>> size=1758786048 >> >>>>> dsa >> >>>>> >> >>>>> Note that the line >> >>>>> checksum=905cec879afd9c9d22ecc8036131a180 >> >>>>> is common to both, you will also find this value in the db in table >> >>>>> cloud.vm_template >> >>>>> This is the link to the template on disk, and the one referenced in >> >>>>>the >> >>>>> DB >> >>>>> >> >>>>> You can also see from the dates within the two properties files, >> that >> >>>>> this template was originally deployed back in may, but when I added >> >>>>>the 2nd >> >>>>> Secondary Storage Node in November, the template was automatically >> >>>>>copied >> >>>>> across. >> >>>>> >> >>>>> Hopefully all this info helps you. >> >>>>> >> >>>>> Regards >> >>>>> >> >>>>> Geoff Higginbottom >> >>>>> >> >>>>> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 >> >>>>> >> >>>>> geoff.higginbot...@shapeblue.com >> >>>>> >> >>>>> -Original Message- >> >>>>> From: Andrija Panic [mailto:andrija.pa...@gmail.com] >> >>>>> Sent: 13 December 2013 16:47 >> >>>>> To: users@cloudstack.apache.org >> >>>>> Subject: Re: Merging/Removing Secondary Storage NFS >> >>>>> >> >>>>> Hi Jeff, thanks for that detailed explanation, I'm on the KVM by the >> >>>>> way... >> >>>>> For the *volume* snaphost, I can see that real snapshot exist localy >> >>>>>(I >> >>>>> say real snaphost, not clone), but also the clone is created on 1 of >> >>>>>those >> >>>>> NFS servers... >> >>>>> >> >>>>> Can you somehow instruct me to check if new template I have just >> >>>>> created, is actualy copied on all 3 NFS hosts (all3 NFS are online, >> >>>>>and >> >>>>> working >> >>>>> fine) ? >> >>>>> Template is only 4GB of real size on the host, and it should be very >> >>>>> quickly copied to where it shoyld be. (my IO and netowrk traffic are >> >>>>>pretty >> >>>>> low in general). >> >>>>> >> >>>>> Should the templates files be created on all 3 NFS storages, with >> the >> >>>>> same file name? Or can I see this somehow from the database...I'm >> >>>>>trying to >> >>>>> decode it by myself, but can't figure this out... >> >>>>> >> >>>>> Thanks again, >> >>>>> Andrija >> >>>>> >> >>>>> >> >>>>
Re: Merging/Removing Secondary Storage NFS
;>>> #Wed Nov 21 23:14:59 UTC 2012 > >>>>> filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd > >>>>> vhd=true > >>>>> id=5 > >>>>> vhd.filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd > >>>>> public=true > >>>>> uniquename=centos56-x86_64-xen > >>>>> vhd.virtualsize=21474836480 > >>>>> virtualsize=21474836480 > >>>>> checksum=905cec879afd9c9d22ecc8036131a180 > >>>>> hvm=false > >>>>> description=CentOS 5.6(64-bit) no GUI (XenServer) > >>>>> vhd.size=1758786048 > >>>>> size=1758786048 > >>>>> dsa > >>>>> > >>>>> Note that the line > >>>>> checksum=905cec879afd9c9d22ecc8036131a180 > >>>>> is common to both, you will also find this value in the db in table > >>>>> cloud.vm_template > >>>>> This is the link to the template on disk, and the one referenced in > >>>>>the > >>>>> DB > >>>>> > >>>>> You can also see from the dates within the two properties files, that > >>>>> this template was originally deployed back in may, but when I added > >>>>>the 2nd > >>>>> Secondary Storage Node in November, the template was automatically > >>>>>copied > >>>>> across. > >>>>> > >>>>> Hopefully all this info helps you. > >>>>> > >>>>> Regards > >>>>> > >>>>> Geoff Higginbottom > >>>>> > >>>>> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 > >>>>> > >>>>> geoff.higginbot...@shapeblue.com > >>>>> > >>>>> -Original Message- > >>>>> From: Andrija Panic [mailto:andrija.pa...@gmail.com] > >>>>> Sent: 13 December 2013 16:47 > >>>>> To: users@cloudstack.apache.org > >>>>> Subject: Re: Merging/Removing Secondary Storage NFS > >>>>> > >>>>> Hi Jeff, thanks for that detailed explanation, I'm on the KVM by the > >>>>> way... > >>>>> For the *volume* snaphost, I can see that real snapshot exist localy > >>>>>(I > >>>>> say real snaphost, not clone), but also the clone is created on 1 of > >>>>>those > >>>>> NFS servers... > >>>>> > >>>>> Can you somehow instruct me to check if new template I have just > >>>>> created, is actualy copied on all 3 NFS hosts (all3 NFS are online, > >>>>>and > >>>>> working > >>>>> fine) ? > >>>>> Template is only 4GB of real size on the host, and it should be very > >>>>> quickly copied to where it shoyld be. (my IO and netowrk traffic are > >>>>>pretty > >>>>> low in general). > >>>>> > >>>>> Should the templates files be created on all 3 NFS storages, with the > >>>>> same file name? Or can I see this somehow from the database...I'm > >>>>>trying to > >>>>> decode it by myself, but can't figure this out... > >>>>> > >>>>> Thanks again, > >>>>> Andrija > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On 13 December 2013 17:35, Geoff Higginbottom < > >>>>> geoff.higginbot...@shapeblue.com> wrote: > >>>>> > >>>>> > Andrija, > >>>>> > > >>>>> > The design is that any Public Template, or ISO gets copied to every > >>>>> > secondary storage in the zone. The copying is undertaken by the > >>>>>SSVM > >>>>> > and will obviously take some time if you have just added a 2nd > >>>>>Public > >>>>> > ISO or Template, so your copy process may not have finished. > >>>>> > > >>>>> > For 4.2 'Snapshots' are now even more confusing than ever. We now > >>>>> have: > >>>>> > > >>>>> > 1. VM Snapshots (ESXi & XenServer Only) these are true Hypervisor > >>>>> > Snapshots, enabling quick roll back to the Snapsh
Re: Merging/Removing Secondary Storage NFS
This is definitely not expected. But, in the thread below I see you writing that the restarting MS initiated template copy. Is that not the case ? On 16/12/13 11:58 AM, "Andrija Panic" wrote: >Well, only ISOs have been copied to the new NFS storage... no other >templates... > >Is this expected behaviour or should I consider this a bug ? > >Any clue on troubleshooting this ? > > >On 16 December 2013 17:05, Andrija Panic wrote: > >> I have just restarted the management server, and the templates copy >> process has begin to the new NFS server... >> >> Will update. >> >> >> On 16 December 2013 16:14, Andrija Panic >>wrote: >> >>> HI Geoff, >>> I have just added new NFS secondary storage to CS. >>> Can you please tell, when shall I expect all existing ISOs and >>>templates >>> to be copied over (start being copied...) - so far, I can observe NO >>> network traffic. >>> Is this maybe configurable like purge thread, or similar ? I have also >>> issues the script for checking the SSVM ( >>> /usr/local/cloud/systemvm/ssvm-check.sh) and have no problems so far. >>> >>> More important, I can confirm that I do NOT have checksum inside the >>> "template.properties" on public templates, only on system templates... >>> And I can confirm that I do not have public templates being replicated >>>on >>> all NFS storages...that is strange. ISOs do get replicated, but public >>> templates does not... >>> Is this a bug or what...? I can not observer any errors in management >>> log... >>> >>> Thanks, >>> Andrija >>> >>> >>> >>> On 13 December 2013 20:10, Andrija Panic >>>wrote: >>> >>>> Thank you very much, I will check... that... funny thing, I search for >>>> files created in last i.e. 1 hour, and there is only one file created >>>>on 1 >>>> NFS, no files created on other NFSs. >>>> >>>> Thanks >>>> >>>> Sent from Google Nexus 4 >>>> On Dec 13, 2013 6:06 PM, "Geoff Higginbottom" < >>>> geoff.higginbot...@shapeblue.com> wrote: >>>> >>>>> The templates get unique names, however they are in the same folder >>>>> structure. >>>>> >>>>> e.g. >>>>> >>>>> My S1 has >>>>> \\IPADDRESS\lonsec\template\tmpl\1\5 >>>>> >>>>> And my S2 has >>>>> \\IPADDDRESS\lonsec2\template\tmpl\1\5 >>>>> >>>>> Note that the '1' is the account ID of the user who created the >>>>> template, in this case 'system' as this is a system vm template, the >>>>>'5' is >>>>> simply the 5th template created on this system >>>>> >>>>> The two template.properties files contain the following >>>>> >>>>> S1 template.properties file >>>>> # >>>>> #Mon May 20 14:11:06 UTC 2013 >>>>> filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd >>>>> vhd=true >>>>> id=5 >>>>> vhd.filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd >>>>> public=true >>>>> uniquename=centos56-x86_64-xen >>>>> vhd.virtualsize=21474836480 >>>>> virtualsize=21474836480 >>>>> checksum=905cec879afd9c9d22ecc8036131a180 >>>>> hvm=false >>>>> description=CentOS 5.6(64-bit) no GUI (XenServer) >>>>> vhd.size=1758786048 >>>>> size=1758786048 >>>>> >>>>> S1 template.properties file >>>>> # >>>>> #Wed Nov 21 23:14:59 UTC 2012 >>>>> filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd >>>>> vhd=true >>>>> id=5 >>>>> vhd.filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd >>>>> public=true >>>>> uniquename=centos56-x86_64-xen >>>>> vhd.virtualsize=21474836480 >>>>> virtualsize=21474836480 >>>>> checksum=905cec879afd9c9d22ecc8036131a180 >>>>> hvm=false >>>>> description=CentOS 5.6(64-bit) no GUI (XenServer) >>>>> vhd.size=1758786048 >>>>> size=1758786048 >>>>> dsa >>>>> >>>>> Note that the line >>>>> checksum=905cec879afd9c9d22ecc8036131a180 >>>>> is common to both, you will also
Re: Merging/Removing Secondary Storage NFS
Well, only ISOs have been copied to the new NFS storage... no other templates... Is this expected behaviour or should I consider this a bug ? Any clue on troubleshooting this ? On 16 December 2013 17:05, Andrija Panic wrote: > I have just restarted the management server, and the templates copy > process has begin to the new NFS server... > > Will update. > > > On 16 December 2013 16:14, Andrija Panic wrote: > >> HI Geoff, >> I have just added new NFS secondary storage to CS. >> Can you please tell, when shall I expect all existing ISOs and templates >> to be copied over (start being copied...) - so far, I can observe NO >> network traffic. >> Is this maybe configurable like purge thread, or similar ? I have also >> issues the script for checking the SSVM ( >> /usr/local/cloud/systemvm/ssvm-check.sh) and have no problems so far. >> >> More important, I can confirm that I do NOT have checksum inside the >> "template.properties" on public templates, only on system templates... >> And I can confirm that I do not have public templates being replicated on >> all NFS storages...that is strange. ISOs do get replicated, but public >> templates does not... >> Is this a bug or what...? I can not observer any errors in management >> log... >> >> Thanks, >> Andrija >> >> >> >> On 13 December 2013 20:10, Andrija Panic wrote: >> >>> Thank you very much, I will check... that... funny thing, I search for >>> files created in last i.e. 1 hour, and there is only one file created on 1 >>> NFS, no files created on other NFSs. >>> >>> Thanks >>> >>> Sent from Google Nexus 4 >>> On Dec 13, 2013 6:06 PM, "Geoff Higginbottom" < >>> geoff.higginbot...@shapeblue.com> wrote: >>> >>>> The templates get unique names, however they are in the same folder >>>> structure. >>>> >>>> e.g. >>>> >>>> My S1 has >>>> \\IPADDRESS\lonsec\template\tmpl\1\5 >>>> >>>> And my S2 has >>>> \\IPADDDRESS\lonsec2\template\tmpl\1\5 >>>> >>>> Note that the '1' is the account ID of the user who created the >>>> template, in this case 'system' as this is a system vm template, the '5' is >>>> simply the 5th template created on this system >>>> >>>> The two template.properties files contain the following >>>> >>>> S1 template.properties file >>>> # >>>> #Mon May 20 14:11:06 UTC 2013 >>>> filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd >>>> vhd=true >>>> id=5 >>>> vhd.filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd >>>> public=true >>>> uniquename=centos56-x86_64-xen >>>> vhd.virtualsize=21474836480 >>>> virtualsize=21474836480 >>>> checksum=905cec879afd9c9d22ecc8036131a180 >>>> hvm=false >>>> description=CentOS 5.6(64-bit) no GUI (XenServer) >>>> vhd.size=1758786048 >>>> size=1758786048 >>>> >>>> S1 template.properties file >>>> # >>>> #Wed Nov 21 23:14:59 UTC 2012 >>>> filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd >>>> vhd=true >>>> id=5 >>>> vhd.filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd >>>> public=true >>>> uniquename=centos56-x86_64-xen >>>> vhd.virtualsize=21474836480 >>>> virtualsize=21474836480 >>>> checksum=905cec879afd9c9d22ecc8036131a180 >>>> hvm=false >>>> description=CentOS 5.6(64-bit) no GUI (XenServer) >>>> vhd.size=1758786048 >>>> size=1758786048 >>>> dsa >>>> >>>> Note that the line >>>> checksum=905cec879afd9c9d22ecc8036131a180 >>>> is common to both, you will also find this value in the db in table >>>> cloud.vm_template >>>> This is the link to the template on disk, and the one referenced in the >>>> DB >>>> >>>> You can also see from the dates within the two properties files, that >>>> this template was originally deployed back in may, but when I added the 2nd >>>> Secondary Storage Node in November, the template was automatically copied >>>> across. >>>> >>>> Hopefully all this info helps you. >>>> >>>> Regards >>>> >>>> Geoff Higginbottom >>>> >>>> D: +44 20 3603 0542 |
Re: Merging/Removing Secondary Storage NFS
I have just restarted the management server, and the templates copy process has begin to the new NFS server... Will update. On 16 December 2013 16:14, Andrija Panic wrote: > HI Geoff, > I have just added new NFS secondary storage to CS. > Can you please tell, when shall I expect all existing ISOs and templates > to be copied over (start being copied...) - so far, I can observe NO > network traffic. > Is this maybe configurable like purge thread, or similar ? I have also > issues the script for checking the SSVM ( > /usr/local/cloud/systemvm/ssvm-check.sh) and have no problems so far. > > More important, I can confirm that I do NOT have checksum inside the > "template.properties" on public templates, only on system templates... > And I can confirm that I do not have public templates being replicated on > all NFS storages...that is strange. ISOs do get replicated, but public > templates does not... > Is this a bug or what...? I can not observer any errors in management > log... > > Thanks, > Andrija > > > > On 13 December 2013 20:10, Andrija Panic wrote: > >> Thank you very much, I will check... that... funny thing, I search for >> files created in last i.e. 1 hour, and there is only one file created on 1 >> NFS, no files created on other NFSs. >> >> Thanks >> >> Sent from Google Nexus 4 >> On Dec 13, 2013 6:06 PM, "Geoff Higginbottom" < >> geoff.higginbot...@shapeblue.com> wrote: >> >>> The templates get unique names, however they are in the same folder >>> structure. >>> >>> e.g. >>> >>> My S1 has >>> \\IPADDRESS\lonsec\template\tmpl\1\5 >>> >>> And my S2 has >>> \\IPADDDRESS\lonsec2\template\tmpl\1\5 >>> >>> Note that the '1' is the account ID of the user who created the >>> template, in this case 'system' as this is a system vm template, the '5' is >>> simply the 5th template created on this system >>> >>> The two template.properties files contain the following >>> >>> S1 template.properties file >>> # >>> #Mon May 20 14:11:06 UTC 2013 >>> filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd >>> vhd=true >>> id=5 >>> vhd.filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd >>> public=true >>> uniquename=centos56-x86_64-xen >>> vhd.virtualsize=21474836480 >>> virtualsize=21474836480 >>> checksum=905cec879afd9c9d22ecc8036131a180 >>> hvm=false >>> description=CentOS 5.6(64-bit) no GUI (XenServer) >>> vhd.size=1758786048 >>> size=1758786048 >>> >>> S1 template.properties file >>> # >>> #Wed Nov 21 23:14:59 UTC 2012 >>> filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd >>> vhd=true >>> id=5 >>> vhd.filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd >>> public=true >>> uniquename=centos56-x86_64-xen >>> vhd.virtualsize=21474836480 >>> virtualsize=21474836480 >>> checksum=905cec879afd9c9d22ecc8036131a180 >>> hvm=false >>> description=CentOS 5.6(64-bit) no GUI (XenServer) >>> vhd.size=1758786048 >>> size=1758786048 >>> dsa >>> >>> Note that the line >>> checksum=905cec879afd9c9d22ecc8036131a180 >>> is common to both, you will also find this value in the db in table >>> cloud.vm_template >>> This is the link to the template on disk, and the one referenced in the >>> DB >>> >>> You can also see from the dates within the two properties files, that >>> this template was originally deployed back in may, but when I added the 2nd >>> Secondary Storage Node in November, the template was automatically copied >>> across. >>> >>> Hopefully all this info helps you. >>> >>> Regards >>> >>> Geoff Higginbottom >>> >>> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 >>> >>> geoff.higginbot...@shapeblue.com >>> >>> -Original Message- >>> From: Andrija Panic [mailto:andrija.pa...@gmail.com] >>> Sent: 13 December 2013 16:47 >>> To: users@cloudstack.apache.org >>> Subject: Re: Merging/Removing Secondary Storage NFS >>> >>> Hi Jeff, thanks for that detailed explanation, I'm on the KVM by the >>> way... >>> For the *volume* snaphost, I can see that real snapshot exist localy (I >>> say real snaphost, not clone), but also the clone is created on 1 of those >>> NFS servers..
Re: Merging/Removing Secondary Storage NFS
HI Geoff, I have just added new NFS secondary storage to CS. Can you please tell, when shall I expect all existing ISOs and templates to be copied over (start being copied...) - so far, I can observe NO network traffic. Is this maybe configurable like purge thread, or similar ? I have also issues the script for checking the SSVM ( /usr/local/cloud/systemvm/ssvm-check.sh) and have no problems so far. More important, I can confirm that I do NOT have checksum inside the "template.properties" on public templates, only on system templates... And I can confirm that I do not have public templates being replicated on all NFS storages...that is strange. ISOs do get replicated, but public templates does not... Is this a bug or what...? I can not observer any errors in management log... Thanks, Andrija On 13 December 2013 20:10, Andrija Panic wrote: > Thank you very much, I will check... that... funny thing, I search for > files created in last i.e. 1 hour, and there is only one file created on 1 > NFS, no files created on other NFSs. > > Thanks > > Sent from Google Nexus 4 > On Dec 13, 2013 6:06 PM, "Geoff Higginbottom" < > geoff.higginbot...@shapeblue.com> wrote: > >> The templates get unique names, however they are in the same folder >> structure. >> >> e.g. >> >> My S1 has >> \\IPADDRESS\lonsec\template\tmpl\1\5 >> >> And my S2 has >> \\IPADDDRESS\lonsec2\template\tmpl\1\5 >> >> Note that the '1' is the account ID of the user who created the template, >> in this case 'system' as this is a system vm template, the '5' is simply >> the 5th template created on this system >> >> The two template.properties files contain the following >> >> S1 template.properties file >> # >> #Mon May 20 14:11:06 UTC 2013 >> filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd >> vhd=true >> id=5 >> vhd.filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd >> public=true >> uniquename=centos56-x86_64-xen >> vhd.virtualsize=21474836480 >> virtualsize=21474836480 >> checksum=905cec879afd9c9d22ecc8036131a180 >> hvm=false >> description=CentOS 5.6(64-bit) no GUI (XenServer) >> vhd.size=1758786048 >> size=1758786048 >> >> S1 template.properties file >> # >> #Wed Nov 21 23:14:59 UTC 2012 >> filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd >> vhd=true >> id=5 >> vhd.filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd >> public=true >> uniquename=centos56-x86_64-xen >> vhd.virtualsize=21474836480 >> virtualsize=21474836480 >> checksum=905cec879afd9c9d22ecc8036131a180 >> hvm=false >> description=CentOS 5.6(64-bit) no GUI (XenServer) >> vhd.size=1758786048 >> size=1758786048 >> dsa >> >> Note that the line >> checksum=905cec879afd9c9d22ecc8036131a180 >> is common to both, you will also find this value in the db in table >> cloud.vm_template >> This is the link to the template on disk, and the one referenced in the DB >> >> You can also see from the dates within the two properties files, that >> this template was originally deployed back in may, but when I added the 2nd >> Secondary Storage Node in November, the template was automatically copied >> across. >> >> Hopefully all this info helps you. >> >> Regards >> >> Geoff Higginbottom >> >> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 >> >> geoff.higginbot...@shapeblue.com >> >> -Original Message- >> From: Andrija Panic [mailto:andrija.pa...@gmail.com] >> Sent: 13 December 2013 16:47 >> To: users@cloudstack.apache.org >> Subject: Re: Merging/Removing Secondary Storage NFS >> >> Hi Jeff, thanks for that detailed explanation, I'm on the KVM by the >> way... >> For the *volume* snaphost, I can see that real snapshot exist localy (I >> say real snaphost, not clone), but also the clone is created on 1 of those >> NFS servers... >> >> Can you somehow instruct me to check if new template I have just created, >> is actualy copied on all 3 NFS hosts (all3 NFS are online, and working >> fine) ? >> Template is only 4GB of real size on the host, and it should be very >> quickly copied to where it shoyld be. (my IO and netowrk traffic are pretty >> low in general). >> >> Should the templates files be created on all 3 NFS storages, with the >> same file name? Or can I see this somehow from the database...I'm trying to >> decode it by myself, but can't figure this out... >> >> Thanks again, >> Andrija >&
RE: Merging/Removing Secondary Storage NFS
Thank you very much, I will check... that... funny thing, I search for files created in last i.e. 1 hour, and there is only one file created on 1 NFS, no files created on other NFSs. Thanks Sent from Google Nexus 4 On Dec 13, 2013 6:06 PM, "Geoff Higginbottom" < geoff.higginbot...@shapeblue.com> wrote: > The templates get unique names, however they are in the same folder > structure. > > e.g. > > My S1 has > \\IPADDRESS\lonsec\template\tmpl\1\5 > > And my S2 has > \\IPADDDRESS\lonsec2\template\tmpl\1\5 > > Note that the '1' is the account ID of the user who created the template, > in this case 'system' as this is a system vm template, the '5' is simply > the 5th template created on this system > > The two template.properties files contain the following > > S1 template.properties file > # > #Mon May 20 14:11:06 UTC 2013 > filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd > vhd=true > id=5 > vhd.filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd > public=true > uniquename=centos56-x86_64-xen > vhd.virtualsize=21474836480 > virtualsize=21474836480 > checksum=905cec879afd9c9d22ecc8036131a180 > hvm=false > description=CentOS 5.6(64-bit) no GUI (XenServer) > vhd.size=1758786048 > size=1758786048 > > S1 template.properties file > # > #Wed Nov 21 23:14:59 UTC 2012 > filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd > vhd=true > id=5 > vhd.filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd > public=true > uniquename=centos56-x86_64-xen > vhd.virtualsize=21474836480 > virtualsize=21474836480 > checksum=905cec879afd9c9d22ecc8036131a180 > hvm=false > description=CentOS 5.6(64-bit) no GUI (XenServer) > vhd.size=1758786048 > size=1758786048 > dsa > > Note that the line > checksum=905cec879afd9c9d22ecc8036131a180 > is common to both, you will also find this value in the db in table > cloud.vm_template > This is the link to the template on disk, and the one referenced in the DB > > You can also see from the dates within the two properties files, that this > template was originally deployed back in may, but when I added the 2nd > Secondary Storage Node in November, the template was automatically copied > across. > > Hopefully all this info helps you. > > Regards > > Geoff Higginbottom > > D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 > > geoff.higginbot...@shapeblue.com > > -Original Message- > From: Andrija Panic [mailto:andrija.pa...@gmail.com] > Sent: 13 December 2013 16:47 > To: users@cloudstack.apache.org > Subject: Re: Merging/Removing Secondary Storage NFS > > Hi Jeff, thanks for that detailed explanation, I'm on the KVM by the way... > For the *volume* snaphost, I can see that real snapshot exist localy (I > say real snaphost, not clone), but also the clone is created on 1 of those > NFS servers... > > Can you somehow instruct me to check if new template I have just created, > is actualy copied on all 3 NFS hosts (all3 NFS are online, and working > fine) ? > Template is only 4GB of real size on the host, and it should be very > quickly copied to where it shoyld be. (my IO and netowrk traffic are pretty > low in general). > > Should the templates files be created on all 3 NFS storages, with the same > file name? Or can I see this somehow from the database...I'm trying to > decode it by myself, but can't figure this out... > > Thanks again, > Andrija > > > > > On 13 December 2013 17:35, Geoff Higginbottom < > geoff.higginbot...@shapeblue.com> wrote: > > > Andrija, > > > > The design is that any Public Template, or ISO gets copied to every > > secondary storage in the zone. The copying is undertaken by the SSVM > > and will obviously take some time if you have just added a 2nd Public > > ISO or Template, so your copy process may not have finished. > > > > For 4.2 'Snapshots' are now even more confusing than ever. We now have: > > > > 1. VM Snapshots (ESXi & XenServer Only) these are true Hypervisor > > Snapshots, enabling quick roll back to the Snapshot, including Memory > State. > > 2. Volume Snapshots (better thought of as Volume Backups) > > > > VM Snapshots remain on Primary Storage to enable the quick roll-back > > to Snapshot to work. > > Volume Snapshots (backups) get copied to Secondary Storage. If you > > have more than one Secondary Storage available, the first backup is > > copied to one of the Secondary Storage volumes, chosen at random, then > > all subsequent backups are always copied to the same Secondary Storage > > volume > > > > Regards > > > &g
RE: Merging/Removing Secondary Storage NFS
The templates get unique names, however they are in the same folder structure. e.g. My S1 has \\IPADDRESS\lonsec\template\tmpl\1\5 And my S2 has \\IPADDDRESS\lonsec2\template\tmpl\1\5 Note that the '1' is the account ID of the user who created the template, in this case 'system' as this is a system vm template, the '5' is simply the 5th template created on this system The two template.properties files contain the following S1 template.properties file # #Mon May 20 14:11:06 UTC 2013 filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd vhd=true id=5 vhd.filename=d015ebd8-5395-31a5-9648-a179062885d8.vhd public=true uniquename=centos56-x86_64-xen vhd.virtualsize=21474836480 virtualsize=21474836480 checksum=905cec879afd9c9d22ecc8036131a180 hvm=false description=CentOS 5.6(64-bit) no GUI (XenServer) vhd.size=1758786048 size=1758786048 S1 template.properties file # #Wed Nov 21 23:14:59 UTC 2012 filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd vhd=true id=5 vhd.filename=8b106378-7d6d-31b3-aab7-bfc85ef97884.vhd public=true uniquename=centos56-x86_64-xen vhd.virtualsize=21474836480 virtualsize=21474836480 checksum=905cec879afd9c9d22ecc8036131a180 hvm=false description=CentOS 5.6(64-bit) no GUI (XenServer) vhd.size=1758786048 size=1758786048 dsa Note that the line checksum=905cec879afd9c9d22ecc8036131a180 is common to both, you will also find this value in the db in table cloud.vm_template This is the link to the template on disk, and the one referenced in the DB You can also see from the dates within the two properties files, that this template was originally deployed back in may, but when I added the 2nd Secondary Storage Node in November, the template was automatically copied across. Hopefully all this info helps you. Regards Geoff Higginbottom D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 geoff.higginbot...@shapeblue.com -Original Message- From: Andrija Panic [mailto:andrija.pa...@gmail.com] Sent: 13 December 2013 16:47 To: users@cloudstack.apache.org Subject: Re: Merging/Removing Secondary Storage NFS Hi Jeff, thanks for that detailed explanation, I'm on the KVM by the way... For the *volume* snaphost, I can see that real snapshot exist localy (I say real snaphost, not clone), but also the clone is created on 1 of those NFS servers... Can you somehow instruct me to check if new template I have just created, is actualy copied on all 3 NFS hosts (all3 NFS are online, and working fine) ? Template is only 4GB of real size on the host, and it should be very quickly copied to where it shoyld be. (my IO and netowrk traffic are pretty low in general). Should the templates files be created on all 3 NFS storages, with the same file name? Or can I see this somehow from the database...I'm trying to decode it by myself, but can't figure this out... Thanks again, Andrija On 13 December 2013 17:35, Geoff Higginbottom < geoff.higginbot...@shapeblue.com> wrote: > Andrija, > > The design is that any Public Template, or ISO gets copied to every > secondary storage in the zone. The copying is undertaken by the SSVM > and will obviously take some time if you have just added a 2nd Public > ISO or Template, so your copy process may not have finished. > > For 4.2 'Snapshots' are now even more confusing than ever. We now have: > > 1. VM Snapshots (ESXi & XenServer Only) these are true Hypervisor > Snapshots, enabling quick roll back to the Snapshot, including Memory State. > 2. Volume Snapshots (better thought of as Volume Backups) > > VM Snapshots remain on Primary Storage to enable the quick roll-back > to Snapshot to work. > Volume Snapshots (backups) get copied to Secondary Storage. If you > have more than one Secondary Storage available, the first backup is > copied to one of the Secondary Storage volumes, chosen at random, then > all subsequent backups are always copied to the same Secondary Storage > volume > > Regards > > Geoff Higginbottom > > D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 > > geoff.higginbot...@shapeblue.com > > -----Original Message----- > From: Andrija Panic [mailto:andrija.pa...@gmail.com] > Sent: 13 December 2013 16:29 > To: users@cloudstack.apache.org > Subject: Re: Merging/Removing Secondary Storage NFS > > HI Geoff, > But, I have verified that my public templates are storaged only on 1 > host... (at least based on file name of the template on one of those > host - these file names does NOT exist on other 2 NFS hosts...) Any clue ? > > Can you also confirm that now in 4.2, since snaphost are really a > snapshots, they are not moved to secondary storage, but exist localy > on the primary storage next to the original image they were created > from (I can see this with lsof and qemu-img info). > > Thanks > > > On 13 Decem
Re: Merging/Removing Secondary Storage NFS
Hi Jeff, thanks for that detailed explanation, I'm on the KVM by the way... For the *volume* snaphost, I can see that real snapshot exist localy (I say real snaphost, not clone), but also the clone is created on 1 of those NFS servers... Can you somehow instruct me to check if new template I have just created, is actualy copied on all 3 NFS hosts (all3 NFS are online, and working fine) ? Template is only 4GB of real size on the host, and it should be very quickly copied to where it shoyld be. (my IO and netowrk traffic are pretty low in general). Should the templates files be created on all 3 NFS storages, with the same file name? Or can I see this somehow from the database...I'm trying to decode it by myself, but can't figure this out... Thanks again, Andrija On 13 December 2013 17:35, Geoff Higginbottom < geoff.higginbot...@shapeblue.com> wrote: > Andrija, > > The design is that any Public Template, or ISO gets copied to every > secondary storage in the zone. The copying is undertaken by the SSVM and > will obviously take some time if you have just added a 2nd Public ISO or > Template, so your copy process may not have finished. > > For 4.2 'Snapshots' are now even more confusing than ever. We now have: > > 1. VM Snapshots (ESXi & XenServer Only) these are true Hypervisor > Snapshots, enabling quick roll back to the Snapshot, including Memory State. > 2. Volume Snapshots (better thought of as Volume Backups) > > VM Snapshots remain on Primary Storage to enable the quick roll-back to > Snapshot to work. > Volume Snapshots (backups) get copied to Secondary Storage. If you have > more than one Secondary Storage available, the first backup is copied to > one of the Secondary Storage volumes, chosen at random, then all subsequent > backups are always copied to the same Secondary Storage volume > > Regards > > Geoff Higginbottom > > D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 > > geoff.higginbot...@shapeblue.com > > -Original Message- > From: Andrija Panic [mailto:andrija.pa...@gmail.com] > Sent: 13 December 2013 16:29 > To: users@cloudstack.apache.org > Subject: Re: Merging/Removing Secondary Storage NFS > > HI Geoff, > But, I have verified that my public templates are storaged only on 1 > host... (at least based on file name of the template on one of those host - > these file names does NOT exist on other 2 NFS hosts...) Any clue ? > > Can you also confirm that now in 4.2, since snaphost are really a > snapshots, they are not moved to secondary storage, but exist localy on the > primary storage next to the original image they were created from (I can > see this with lsof and qemu-img info). > > Thanks > > > On 13 December 2013 17:21, Geoff Higginbottom < > geoff.higginbot...@shapeblue.com> wrote: > > > You don’t need to copy Public Templates from S1 to S2 if they have > > both been on line as Public Templates are always copied to every sec > > storage in the zone > > > > Regards > > > > Geoff Higginbottom > > > > D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 > > > > geoff.higginbot...@shapeblue.com > > > > -Original Message- > > From: Andrija Panic [mailto:andrija.pa...@gmail.com] > > Sent: 13 December 2013 16:11 > > To: users@cloudstack.apache.org > > Subject: Re: Merging/Removing Secondary Storage NFS > > > > Just to correct myself: that guide: tells nice job for migrating > > snapshots. > > > > But I'm confused why only private templates should be migrated, why > > not also the public ones? > > > > Quote from the page: > > *"Note*: You must copy only private templates on Secondary storage > > host S2 to S1." > > And the whole guide is about migrating those private templates (these > > are mostly system VM templates and such) > > > > What about public templates? I have predefined some public templates, > > that as far as I can see, are only storaged on some of my 3 NFS hosts... > > ? > > > > Thanks > > > > > > > > On 13 December 2013 16:51, Andrija Panic > wrote: > > > > > HI, > > > > > > anyone has good doc/procedure for removing old NFS (migrate > > > everything to new NFS server) ? > > > > > > I was thinking of editing the database - dump to sql file, search, > > > replace old IP address with new IP address, and rsync everything > > > from old NFS to new NFS, keeping the same directory structure. > > > Then import the edited dump file. > > > > > > I also found some Citrix's guide, but that does not
RE: Merging/Removing Secondary Storage NFS
Andrija, The design is that any Public Template, or ISO gets copied to every secondary storage in the zone. The copying is undertaken by the SSVM and will obviously take some time if you have just added a 2nd Public ISO or Template, so your copy process may not have finished. For 4.2 'Snapshots' are now even more confusing than ever. We now have: 1. VM Snapshots (ESXi & XenServer Only) these are true Hypervisor Snapshots, enabling quick roll back to the Snapshot, including Memory State. 2. Volume Snapshots (better thought of as Volume Backups) VM Snapshots remain on Primary Storage to enable the quick roll-back to Snapshot to work. Volume Snapshots (backups) get copied to Secondary Storage. If you have more than one Secondary Storage available, the first backup is copied to one of the Secondary Storage volumes, chosen at random, then all subsequent backups are always copied to the same Secondary Storage volume Regards Geoff Higginbottom D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 geoff.higginbot...@shapeblue.com -Original Message- From: Andrija Panic [mailto:andrija.pa...@gmail.com] Sent: 13 December 2013 16:29 To: users@cloudstack.apache.org Subject: Re: Merging/Removing Secondary Storage NFS HI Geoff, But, I have verified that my public templates are storaged only on 1 host... (at least based on file name of the template on one of those host - these file names does NOT exist on other 2 NFS hosts...) Any clue ? Can you also confirm that now in 4.2, since snaphost are really a snapshots, they are not moved to secondary storage, but exist localy on the primary storage next to the original image they were created from (I can see this with lsof and qemu-img info). Thanks On 13 December 2013 17:21, Geoff Higginbottom < geoff.higginbot...@shapeblue.com> wrote: > You don’t need to copy Public Templates from S1 to S2 if they have > both been on line as Public Templates are always copied to every sec > storage in the zone > > Regards > > Geoff Higginbottom > > D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 > > geoff.higginbot...@shapeblue.com > > -Original Message- > From: Andrija Panic [mailto:andrija.pa...@gmail.com] > Sent: 13 December 2013 16:11 > To: users@cloudstack.apache.org > Subject: Re: Merging/Removing Secondary Storage NFS > > Just to correct myself: that guide: tells nice job for migrating > snapshots. > > But I'm confused why only private templates should be migrated, why > not also the public ones? > > Quote from the page: > *"Note*: You must copy only private templates on Secondary storage > host S2 to S1." > And the whole guide is about migrating those private templates (these > are mostly system VM templates and such) > > What about public templates? I have predefined some public templates, > that as far as I can see, are only storaged on some of my 3 NFS hosts... > ? > > Thanks > > > > On 13 December 2013 16:51, Andrija Panic wrote: > > > HI, > > > > anyone has good doc/procedure for removing old NFS (migrate > > everything to new NFS server) ? > > > > I was thinking of editing the database - dump to sql file, search, > > replace old IP address with new IP address, and rsync everything > > from old NFS to new NFS, keeping the same directory structure. > > Then import the edited dump file. > > > > I also found some Citrix's guide, but that does not seem to migrate > > everything (only the system VM templates, as per my understanding > > after reading the docs...) > > http://support.citrix.com/article/CTX135229 > > > > Thanks. > > > > -- > > > > Andrija Panić > > -- > > http://admintweets.com > > -- > > > > > > -- > > Andrija Panić > -- > http://admintweets.com > -- > This email and any attachments to it may be confidential and are > intended solely for the use of the individual to whom it is addressed. > Any views or opinions expressed are solely those of the author and do > not necessarily represent those of Shape Blue Ltd or related > companies. If you are not the intended recipient of this email, you > must neither take any action based upon its contents, nor copy or show > it to anyone. Please contact the sender if you believe you have > received this email in error. Shape Blue Ltd is a company incorporated > in England & Wales. ShapeBlue Services India LLP is a company > incorporated in India and is operated under license from Shape Blue > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorpora
Re: Merging/Removing Secondary Storage NFS
HI Geoff, But, I have verified that my public templates are storaged only on 1 host... (at least based on file name of the template on one of those host - these file names does NOT exist on other 2 NFS hosts...) Any clue ? Can you also confirm that now in 4.2, since snaphost are really a snapshots, they are not moved to secondary storage, but exist localy on the primary storage next to the original image they were created from (I can see this with lsof and qemu-img info). Thanks On 13 December 2013 17:21, Geoff Higginbottom < geoff.higginbot...@shapeblue.com> wrote: > You don’t need to copy Public Templates from S1 to S2 if they have both > been on line as Public Templates are always copied to every sec storage in > the zone > > Regards > > Geoff Higginbottom > > D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 > > geoff.higginbot...@shapeblue.com > > -Original Message- > From: Andrija Panic [mailto:andrija.pa...@gmail.com] > Sent: 13 December 2013 16:11 > To: users@cloudstack.apache.org > Subject: Re: Merging/Removing Secondary Storage NFS > > Just to correct myself: that guide: tells nice job for migrating > snapshots. > > But I'm confused why only private templates should be migrated, why not > also the public ones? > > Quote from the page: > *"Note*: You must copy only private templates on Secondary storage host S2 > to S1." > And the whole guide is about migrating those private templates (these are > mostly system VM templates and such) > > What about public templates? I have predefined some public templates, that > as far as I can see, are only storaged on some of my 3 NFS hosts... > ? > > Thanks > > > > On 13 December 2013 16:51, Andrija Panic wrote: > > > HI, > > > > anyone has good doc/procedure for removing old NFS (migrate everything > > to new NFS server) ? > > > > I was thinking of editing the database - dump to sql file, search, > > replace old IP address with new IP address, and rsync everything from > > old NFS to new NFS, keeping the same directory structure. > > Then import the edited dump file. > > > > I also found some Citrix's guide, but that does not seem to migrate > > everything (only the system VM templates, as per my understanding > > after reading the docs...) http://support.citrix.com/article/CTX135229 > > > > Thanks. > > > > -- > > > > Andrija Panić > > -- > > http://admintweets.com > > -- > > > > > > -- > > Andrija Panić > -- > http://admintweets.com > -- > This email and any attachments to it may be confidential and are intended > solely for the use of the individual to whom it is addressed. Any views or > opinions expressed are solely those of the author and do not necessarily > represent those of Shape Blue Ltd or related companies. If you are not the > intended recipient of this email, you must neither take any action based > upon its contents, nor copy or show it to anyone. Please contact the sender > if you believe you have received this email in error. Shape Blue Ltd is a > company incorporated in England & Wales. ShapeBlue Services India LLP is a > company incorporated in India and is operated under license from Shape Blue > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil > and is operated under license from Shape Blue Ltd. ShapeBlue is a > registered trademark. > -- Andrija Panić -- http://admintweets.com --
RE: Merging/Removing Secondary Storage NFS
You don’t need to copy Public Templates from S1 to S2 if they have both been on line as Public Templates are always copied to every sec storage in the zone Regards Geoff Higginbottom D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581 geoff.higginbot...@shapeblue.com -Original Message- From: Andrija Panic [mailto:andrija.pa...@gmail.com] Sent: 13 December 2013 16:11 To: users@cloudstack.apache.org Subject: Re: Merging/Removing Secondary Storage NFS Just to correct myself: that guide: tells nice job for migrating snapshots. But I'm confused why only private templates should be migrated, why not also the public ones? Quote from the page: *"Note*: You must copy only private templates on Secondary storage host S2 to S1." And the whole guide is about migrating those private templates (these are mostly system VM templates and such) What about public templates? I have predefined some public templates, that as far as I can see, are only storaged on some of my 3 NFS hosts... ? Thanks On 13 December 2013 16:51, Andrija Panic wrote: > HI, > > anyone has good doc/procedure for removing old NFS (migrate everything > to new NFS server) ? > > I was thinking of editing the database - dump to sql file, search, > replace old IP address with new IP address, and rsync everything from > old NFS to new NFS, keeping the same directory structure. > Then import the edited dump file. > > I also found some Citrix's guide, but that does not seem to migrate > everything (only the system VM templates, as per my understanding > after reading the docs...) http://support.citrix.com/article/CTX135229 > > Thanks. > > -- > > Andrija Panić > -- > http://admintweets.com > -- > -- Andrija Panić -- http://admintweets.com -- This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: Merging/Removing Secondary Storage NFS
Just to correct myself: that guide: tells nice job for migrating snapshots. But I'm confused why only private templates should be migrated, why not also the public ones? Quote from the page: *"Note*: You must copy only private templates on Secondary storage host S2 to S1." And the whole guide is about migrating those private templates (these are mostly system VM templates and such) What about public templates? I have predefined some public templates, that as far as I can see, are only storaged on some of my 3 NFS hosts... ? Thanks On 13 December 2013 16:51, Andrija Panic wrote: > HI, > > anyone has good doc/procedure for removing old NFS (migrate everything to > new NFS server) ? > > I was thinking of editing the database - dump to sql file, search, replace > old IP address with new IP address, and rsync everything from old NFS to > new NFS, keeping the same directory structure. > Then import the edited dump file. > > I also found some Citrix's guide, but that does not seem to migrate > everything (only the system VM templates, as per my understanding after > reading the docs...) http://support.citrix.com/article/CTX135229 > > Thanks. > > -- > > Andrija Panić > -- > http://admintweets.com > -- > -- Andrija Panić -- http://admintweets.com --
